Parse Opcodes from Binary Code

Question

I have been using the Distorm library, using the distorm_decode() function in x64 to retrieve Opcode and Operands from binary code. For example, 44 0F45 C8 is decoded to cmovne r9d,eax.

I would like to have a way to parse the hex binary instruction to extract the opcode part of it, in the example 44 *0F45* C8, would like to extract *0F45* which is the cmovne opcode part.

I was looking at here and I was thinking of maybe doing a lookup at a given Hex Opcode, given a mnemonic string.

Are there any other built in libraries like Distorm that can achieve this? As far as I understand, Distorm has a distorm_decompose function and interface, but I can't reach to the cmovne binary representation *0F45* from the struct _DInst returned from that call.

Answer 1

Basically your usual x86_64 Intel CISC instructions have 2 different opcode formats legacy and VEX.

---

Common opcodes

The legacy opcodes which includes the most common ones used consists of:

• 1 - 4 bytes legacy prefixes which are optional
• 1 byte of a REX prefix which is sometimes required
• 1 - 3 bytes of opcode
• 1 - 18 bytes of additional operand/mode data

If I understand correctly, you are only interested in the sequence of the 1 - 3 bytes In order to extract it from a single instruction you need to determine whether the prefixes "ended" and how long the opcode is.

The second point is pretty simple, the opcode pattern is as such: Let # denote a single byte A sequence of bytes is a legacy x86 opcode can only be in one of the following patterns:

• #
• 0x0F # ← This is the pattern in your example
• 0x0F 0x3A #

The second part appears more tricky since there are theoretically many combinations for those 5 prefix bytes (legacy + prefix).

Legacy prefix bytes were used by certain operations to mark another meaning, for example to make an operation on other memory segment, this is a limited set which consists (according to this) of: 0xF0, 0xF2, 0xF3, 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65, 0x66 and 0x67

So, a 1-4 byte sequence of these bytes at the start should be ignored.

The REX prefix is used to operate in long mode and is usually used when 64 bit register operands are present or when an extended register is used R8 - R15. As this is also a prefix, this also should be ignored A REX prefix always starts with the bit sequence 0100 being the most significant nibble of the byte (0x4X).

Althogh it does not cover all instructions this should should suffice for most of the daily x86 instructions you encounter.

---

Example

Let's apply the method above to analyze the instruction: 0x44 0F 45 C8

First of all prefixes, 0x44 isn't in the legacy set, but it is obvious from the most significant (left) nibble 4 digit that this is a REX prefix. (0100 0100 here). Which means we're done with prefixes, next is the opcode, 0x0F and a byte, done matching 2-byte opcode, and it's indeed 0F45 as you can verify. The remaining trailing opcode data is ignored. Done.

The other type of operations is VEX / XOP which are used mainly for vector / floating point operations but their detection is more complex. You can learn more about them here.

Answer 2

Elian posted a nice answer that explains the theory

here is an implementation using capstone and python

import sys
import capstone

print ( "dissecting a stram of hex pairs to its components in x86")

if(len(sys.argv) < 2):
    sys.exit("usage python %s quoted_hex_pairs like %s" % ( sys.argv[0] ,
    "\"F0 0D 15 F0 0D BA D0 12 50 0D\""))

CODE = []
a = sys.argv[1].split(' ')
for i in range(0,len(a),1):
    CODE.append( chr(int(a[i],16)))
CODESTR = ''.join(CODE)

md = capstone.Cs(capstone.CS_ARCH_X86, capstone.CS_MODE_64)
md.detail = True

for i in md.disasm(CODESTR, 0x1000):
    print "TLDR:"
    for j in range( 0,len(i.opcode),1):
        if(i.opcode[j] !=0):
            print "%02x" % i.opcode[j],
    print "\n"            
    print ( "FULL DETAILS"              )
    print ( "i.address",    i.address   );    print ( "i.mnemonic",   i.mnemonic    ) 
    print ( "i.op_str" ,    i.op_str    );    print (   "i.id" ,        i.id        ) 
    print ( "i.size" ,      i.size      );    print (   "i.bytes" ,     i.bytes     ) 
    print ( "i.prefix" ,    i.prefix    );    print (   "i.opcode" ,    i.opcode    )
    print ( "i.rex" ,       i.rex       );    print (   "i.addr_size" , i.addr_size ) 
    print ( "i.modrm" ,     i.modrm     );    print (   "i.sib" ,       i.sib       ) 
    print ( "i.disp" ,      i.disp      );    print (   "i.sib_index" , i.sib_index ) 
    print ( "i.sib_scale" , i.sib_scale );    print (   "i.sib_base" ,  i.sib_base  )
    print ( "i.sse_cc" ,    i.sse_cc    );    print (   "i.avx_cc" ,    i.avx_cc    ) 
    print ( "i.avx_sae" ,   i.avx_sae   );    print (   "i.avx_rm" ,    i.avx_rm    ) 

output

python capstest.py "44 0f 45 c8"
dissecting a stram of hex pairs to its components in x86
TLDR:
0f 45

FULL DETAILS
('i.address', 4096L)
('i.mnemonic', u'cmovne')
('i.op_str', u'r9d, eax')
('i.id', 83L)
('i.size', 4)
('i.bytes', bytearray(b'D\x0fE\xc8'))
('i.prefix', [0, 0, 0, 0])
('i.opcode', [15, 69, 0, 0])
('i.rex', 68)
('i.addr_size', 8)
('i.modrm', 200)
('i.sib', 0)
('i.disp', 0)
('i.sib_index', 0L)
('i.sib_scale', 0)
('i.sib_base', 0L)
('i.sse_cc', 0L)
('i.avx_cc', 0L)
('i.avx_sae', False)
('i.avx_rm', 0L)