tl;dr: What would be alternative approaches for attacking an anti-dump design that allocates encrypted blocks of code on the heap and uses a vm to resolve addresses and decrypt before execution?
I'm working on a binary with about 900 pages (4k) of stolen bytes. The binary in .text jumps with a hardcoded address as a parameter that runs through VMcode, generates a value based on a few identifying features (cpuid, threadid, processid), and eventually decrypts the page on the heap and jumps to it in order to execute the stolen functionality. It doesn't shred the bytes or re-encrypt them most likely for performance reasons - future calls jump directly to the destination code.
I have isolated a few blocks of code that decrypt the code itself and jump (haven't quite figured out the key generation yet), and have considered a few approaches:
1.Dump every 4k page allocated(about 900 or so) and append them to the end of the binary at the right address (they won't be allocated in a dumped binary due to real OEP being after the initial packer OEP that sets all this up), and try to trace the vm to manually fix the jumps in the .text section to the decrypted address.
2.Decrypt the pages, strip out garbage instructions (will probably take a while with not the best confidence level), and paste them in-place where they should be in the executable (the bytes that were stolen have its encrypted counterpart in the original location).
3.Try to inject some shared library at the packer OEP, override ntalloc, allocate the memory in a linear fashion accessible from the dll, and log all the decrypted functions; eventually, write some kind of loader that can use the dumped memory as some kind of switchboard that can jump and update the calls in the original binary to jump into the library that would redirect control flow.
4.Some other method I haven't considered.
It seems like they would all work, but the amount of work in any case seems to range from a lot to a ridiculous amount. Are there any other approaches that might be worth exploring?
Thanks!
dynamic binary instrumentationtool (e.g. Pin) to isolate the instructions used for unpacking. I have used this approach to deal with a self-modifying code challenge (so it may be less sophisticated than yours).