Reverse engineering the virtual machine based crackme

Question

I am currently working on some crackme, that has implemented an obfuscation technique, virtualisation. The virtual machine inside this crackme is a huge switch-case block (over 130 cases in it). I have already read dozens of article, but none are making things clear enough.

I have found some info though, that I can find some buffer in memory, containing opcodes, that are native assembly code that are then interpreted by the virtual machine in switch cases block.

Finding them will provide an opportunity to write a small disassembler on any language preferred, where in "case"-blocks would be the native operations written, like printf("mov eax, dword ptr [ebp-4]"). That way I can figure out the algorithm of generating and checking the serial.

However, I am stuck on finding that memory location with opcodes. Can anyone advice me something on how to find these, some techniques, or at least some good literature or tutorials on how to deal with such a thing? May be there are some common ways to crack this kind of crackme.

Your help is very much appreciated.

Answer 1

First of all, the fact to turn a program into a bytecode that will be interpreted by a crafted VM which will be embedded into the software is a quite well-known technique of obfuscation. There have been numerous writings about it.

If you want to find good pointers about it (and how to solve it in different ways), I would advise you to search for "VM-based obfuscation" on any search engine. It should provide you with a lot of literature about it.

Now, solving this kind of thing require first to identify the internal language of the machine and then to understand the program that is coded into this language. For now, there have been only very few progress in automating this process and human is still heavily required to guide and understand what is going on in the obfuscated program.

Yet, one nice thing you can look at is the fact that the VM itself usually looks like another obfuscation called "CFG-flattening" that can be partially processed automatically (even though, I cannot advise you any public tool about it). Anyway, the VM-based obfuscation and the CFG-flattening obfuscation are somehow very close (the CFG-flattening is a subcase of the VM-based obfuscation), so do not hesitate to look at these two techniques and the tricks that may be associated to it.

Here is a bunch of pointers that you may find relevant:

About VM-based obfuscation

• Creating Code Obfuscation Virtual Machines;
• Unpacking Virtualization Obfuscators;
• The Tigress C Diversifier/Obfuscator;
• Jonathan Salwan's deobfuscation method using symbolic execution (see also these slides);
• Deobfuscation of Virtualization-Obfuscated Software: A Semantics-Based Approach;

Also, this question in this website is also relevant to look at.

About CFG-flattening

• What is a “control-flow flattening” obfuscation technique?;
• Control-Flow Flattening;
• Deobfuscation: recovering an OLLVM-protected program;
• Reverse Engineering Obfuscated Code;

Of course, you may find more useful links by yourself! Feel free to improve the list on your own! I just hope that these ones will help you a bit.