How can I detected virtualized code by say, vmprotect, obsidium or themida just by looking at the assembly code? I know that just looking at the PE section name would help, but I wanted to detect these obfuscators virtualization just by looking at the code. How do I go about that?
1 Answer
That's tricky because obfuscated code often looks similar regardless of whether or not it specifically uses virtualization obfuscation. Most VMs ultimately have a similar high-level structure, as laid out in the following image, where there is a central point of dispatch to which almost all handlers ultimately return execution flow. (Although this is not true for all VMs; in so-called "direct threaded" VMs, the handlers pass control directly to one another instead of using a central dispatcher.)
So, one tempting answer to this question would be to look for a loop that ultimately transfers control into a table with pointers. However, VM protections quite often apply a lot of obfuscation to their interpreters -- dead code insertion, pattern-based obfuscation, constant unfolding, breaking the code up into chunks and interspersing jmp instructions between them -- so even if you are looking at the obfuscated code for a VM, the high level structure from the previous image might not be readily apparent. Another common thing for commercial-grade VMs to do is to obfuscate the "dispatcher" part -- for example, by applying some sort of transformation to the pointers in the table, and/or by replacing the jmp with something like push/ret instead.
Given that it can be difficult to determine this statically, it might be easier to determine it by tracing through the obfuscated code dynamically. If there seems to be a common location that the obfuscated code executes very frequently, that might be the fetch/decode/dispatch portion of the figure above. (Of course, as previously mentioned, direct-threaded VMs won't have this feature.)
answered May 2, 2022 at 5:32
