How to obfuscate x86 assembly code?

Question

For my project, I am performing a kind of checksum operation on a portion of code to protect it and therefore do not want its template to be easily visible and therefore need obfuscation.

I have searched a lot on the net and read papers describing obfuscation definitions, types, etc. But there seems to be no tutorial on obfuscating x86 assembly code. Can anybody suggest a simple algorithm/tool for the same?

I have read about inserting dummy code, changing the order of the instructions and other techniques but they appear to be totally random i.e. there is no end to how much dummy code to insert, etc.

Can somebody at least guide me to the correct approach?

Answer 1

You should not confuse "code protection" and "code obfuscation". "Code protection" techniques target in recognizing code modifications (you mention a checksum) and take suitable means like crashing or delivering wrong results when tampering with the code has been recognized. Anti-Debug measures also belong into this category.

"Code obfuscation" in a binary targets in blinding the investigator (and the disassembler) with awkward and often useless code, ranging from simple jmp chains to more complicated constructs. Most code obfuscation techniques result in some code bloat, and the tradeoff between the obfuscation and the performance must be considered. Here are two obfuscation examples being found "in the wild".

• Dummy code: There are many means to insert nops in binaries. Examples range from simple statemens like shr eax, 0 or (in the 32-bit world) shl bx, 20h to more complicated constructions like the following example. The combination of jz and jnz followed by garbage statements (cpuid, ret), avoiding a jmp often confuses disassemblers in their ability to display logical assembly blocks.

mov si, si mov esp, ebp jnz loc_abcd xchg edi, esi mov cl, cl xchg esi, edi mov di, di jz loc_dcba ; followed by a jmp to loc_abcd cpuid ret In the example, the only "real" statement is the mov esp, ebp.

• Complicate simple assembly constructs. You may write a simple jmp as a combination of a push and a ret. Or, if you don't like the "ret" statement, you may replace it by (in 64-bit code): lea rsp, [rsp+8] jmp qword ptr[rsp-8]

Many more, and much more complicated examples can be found. If you want to dive into this matter, you need practice, as in all SW reversing. Get a protected and/or obfuscated binary like a game or a dongle protector and train. In the literature (e.g. the excellent IdaPro book by Chris Eagle) you may find some obfuscation constructs as well.

Have fun!

Answer 2

As mentioned in the comments, check out chapter 5 of the PDF Practical Reverse Engineering for some ideas.

Junk code insertion:

jmp label
<junk> 
label:
<real code>

Operating System–Based Control Indirection:

push addr_seh_handler
push fs:[0]
mov fs:[0], esp
xor eax, eax
mov [eax], 1234h
<junk code>
addr_seh_handler:
<continue execution here>
pop fs:[0]
add esp, 4

Processor-Based Control Indirection:

call target_addr
<junk code>
target_addr:
add esp, 4

Arithmetic Substitution via Identities:

-x = ~x + 1 (by definition of two's complement) 
rotate left(x,y) = (x << y) | (x >> (bits(x)-y))
rotate right(x,y) = (x >> y) | (x << (bits(x)-y))
x-1 = ~-x
x+1 = - x

Pattern-Based Obfuscation:

push reg32

becomes

push imm32
mov dword ptr [esp], reg32

More pattern based examples here.

Nop code insertion

Answer 3

If you don't want to obfuscate the code manually, here's the 'mature' approach:

• Source code -> Compiler -> IR aka bitcode
• IR -> obfuscator -> obfuscated IR
• obfuscated IR -> LLVM static compiler -> final executable

Where:

• The compiler which can generate IR is clang.
• Obfuscator is usually opt from https://github.com/obfuscator-llvm/obfuscator/wiki/Installation
• LLVM static compiler is llc also from https://github.com/obfuscator-llvm/obfuscator/wiki/Installation

Manipulating IR code is much easier, than manipulating the native code. Yet learning how llvm works and how to use its classes to make changes is not trivial.