10

First of all, the fact to turn a program into a bytecode that will be interpreted by a crafted VM which will be embedded into the software is a quite well-known technique of obfuscation. There have been numerous writings about it. If you want to find good pointers about it (and how to solve it in different ways), I would advise you to search for "VM-...


9

Regarding question #1, no IDA does not handle obfuscated binaries. You might be interested by the Virtual Deofbuscator talk Jason Raber gave at Blackhat last year, he also released an IDA plugin and the source code is available. Slides : https://media.blackhat.com/us-13/US-13-Raber-Virtual-Deobfuscator-A-DARPA-Cyber-Fast-Track-Funded-Effort-Slides.pdf ...


4

You should take a look at crackmes.de, especially level 6, like this one, or this one.


3

First, note that most industrial-strength networking devices don't use Linux. If they did, mosts parts of the software would have to be provided in source code to comply with GPL requirements. So while your $100 Linksys router might use Linux, your $3000 Cisco router won't, and it won't be running snort either. (Cisco's IDS and Snort might have several ...


1

Post the data as hex. 100 messages would be great. Are you using ANSI encoding to make patterns in the data easier to see? My guess is there's something like ROT or an xor going on here with messages [monitor] recv() buf: ï|W,7¼≤╛ and [monitor] recv() buf: ïxS*4¬≤╗ being the key the server wants the client to use as that's the first place they change between ...


1

For static analysis, you can use VMAttack plugin for IDA pro. Or if you wanna find how the buffer is built before it's sent you can try ScyllaHide plugin for x64dbg to debug the VmProtected executable. This plugin will help you overcome VMprotect's anti-debug and anti-vm protections. After that simply put a breakpoint on those calls that you hooked and ...


1

One would have to look at the code. How are the jump targets calculated? From your diagrams it could also be something as simple as: void jump_to_x_plus_5(uint x) { jmp x+5; } void jump_to_x_plus_7(uint x) { jmp x+7; } void jump_to_x_plus_9(uint x) { jmp x+9; } And they just replaced all absolute jumps in the .asm with calls to random.choice() of those ...


1

There's now (from 2020) a plugin for IDA Pro that helps in working with obfuscated binaries. It is called D810: Creating an extensible deobfuscation plugin for IDA Pro.


Only top voted, non community-wiki answers of a minimum length are eligible