24

I'm writing a small utility library for hooking functions at run time. I need to find out the length of the first few instructions because I don't want to assume anything or require the developer to manually input the amount of bytes to relocate and overwrite.

There are many great resources to learn assembly but none of them seem to go into much detail on how assembly mnemonics get turned into raw binary instructions.

4
20

If you want to understand the instruction encodings in detail you need to study Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2 (Instruction Set Reference, A-Z). Be aware that Intel IA-32 and AMD64 are very complicated instruction sets and in order to hook a function which is not specifically designed to be hooked by injecting a jump you will run into a great number of different instructions. There is no guarantee that the function even has a stack frame set up.

There are libraries which can do the disassembly and hooking for you, such as Detours by Microsoft Research.

1
  • 1
    Thanks! That seems to be exactly what I'm looking for. Detours is pretty much the same what I'm doing but I'm reinventing the wheel for educational purposes. – Henry Heikkinen Mar 19 '13 at 23:47
10

You can use a disassembler to go from binary opcodes to assembly code.

For example ndisam command is able to do this.

If you have the following binary opcodes (hex view of file):

31C0FFC0C3

You will get the following output when disassembling it with ndisasm:

00000000  31C0              xor ax,ax
00000002  FFC0              inc ax
00000004  C3                ret

Where the first column is the file offset, the second is the binary opcodes and the final row is the assembly code.

You could then get the second column and get the string length of it and divide by 2 and you would have the length of the instruction in bytes.

3
  • While this explains how to do it manually on the command line I don't think it is a suitable approach for a utility library that hooks arbitrary code at runtime. If ndisasm is available as a library it might work though, but then you should add that to your answer. – jix Mar 19 '13 at 20:02
  • I've used both the udis86 and opdis libraries for basically this. Both work reasonably well and will (iirc) expose the length of each instruction as a struct member instead of having to parse asm output. – Sean B. Mar 19 '13 at 20:17
  • @SeanB. taking a quick look at udis86, I can see that it has move to github. – Tyilo Mar 19 '13 at 20:22
6

A lot of people have mentioned the Intel manuals, which are an invaluable reference, but quite hefty. I'd suggest looking at this OSDev wiki page to get an idea of how the instructions are encoded on a simpler level.

For all practical instruction-length-finding problems, I would advise using a disassembler.

Function hooking is an interesting challenge. This MSDN blog explains some of the difficulties well. Depending the requirements, it might be preferable to use the operating system's debugging functionality to attach to the process, "break" on functions, and implement your hook in a separate process.

1
  • That looks like a great overview. Thanks! – Henry Heikkinen Mar 20 '13 at 12:22
4

This CodeProject article is an excellent high-level view of x86's instruction format (with diagrams!). After reading this, more detailed references will make more sense.

Due to many years of backwards-compatible evolution, the x86 instruction format is quite complicated, with all sorts of optional prefixes and instruction-dependent fields, so it is a bit tricky to work out the instruction length. If you want something robust, I would advise adapting existing software rather than rolling your own. But understanding these concepts will of course be very helpful.

3

The ground truth on instruction decoding can be found in the processors manual for software developers. Assembler authors need to know this, so the information is there. For Intel, it's in the beginning of Volume 2A (I think, I lost track since they smushed all the manuals into one PDF). There's a big table that defines how prefixes are encoded, how opcodes are encoded, and how operands are encoded. It's not the easiest reading, but it's there...

1
  • Hmm, when was that? Even just a few weeks ago they were all nicely separated into separate PDF files ... – 0xC0000022L Apr 1 '13 at 1:35
3

The IA-32 Intel® Architecture Software Developer’s Manual Vol. 2 in all its mind-numbing glory.

0
0

You can use the capstone library to disassemble the instructions stating at the address you are looking at. It has the ability to determine for you how long the instruction is in bytes, and then you can use that in your code.

http://www.capstone-engine.org/

0

There are many great resources to learn assembly but none of them seem to go into much detail on how assembly mnemonics get turned into raw binary instructions.

The mnemonics aren't 'turned in' to raw binary instructions, they are the raw binary instructions. It is a human readable representation of the actual hex bytes that are encoding the instructions of the (first generation language) machine code. We typically talk about these bytes by their mnemonics for clarity.

This is unlike assembly (so, second generation language such as gas or masm) or C (third generation language). Both assembly and higher generation languages are source files with a character encoding (such as UTF-8) that encodes the letters of the source code. In the case of assembly, it needs to be assembled to raw x86 bytes. The raw bytes can be disassembled by IDA into a readable form; it's called disassembly because it's presenting the mnemonic representation encoded in a character set like UTF-8 and displayed to the screen (which resembles an assembly language (a second generation language) that would be assembled to machine code -- in this case it's obviously not going to be assembled and isn't really an assembly language on it's own -- it's showing you what the assembly that was compiled to the machine code would have been).

x86 encoding looks like this (I created a diagram):

enter image description here

By the same token, CIL bytecode is a 1st generation language because the opcodes run directly on the virtual machine -- a virtual CPU. This means that the CIL bytecode does not need to be assembled or compiled from a source file with a source character set. ilasm shows a disassembled, mnemonic form of this bytecode.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.