I am interested in reverse engineering a few parts of an .exe application (windows 32-bit executable) that downloads some documents from a server and decrypts them locally in memory.
Using PE EXE identification tools, I found that the application is protected by "VMProtect v3.00 - v3.50".
For some context, I know that VMProtect is a commonly used packer for Windows applications that is notoriously difficult to reverse engineer. In additional to packing .exes, one notable feature of VMProtect is "virtualization" or "VM emulation" where (as far as I understand it) the source code is compiled for a made-up, nonstandard architecture, and a virtual machine for that architecture is generated to allow it to run on Windows. Since the application is compiled for a nonstandard architecture that doesn't actually exist anywhere, it's very difficult to make any progress with reverse engineering it.
Interestingly, the "virtualization" or "VM emulation" feature is optional in VMProtect, and some developers do not enable it because it slows down the final application. In other words, some people use VMProtect simply as a packer, but the unpacked application has standard code that can be decompiled with normal tools.
I found a few guides for unpacking an application that is only packed with VMProtect, without virtualization: https://www.youtube.com/watch?v=aoa89Khfgr0 This seems like it would be difficult for someone with limited experience, like myself, but probably doable.
My question is, how can I tell if virtualization is enabled on the specific application I have? (If it is, I would rather not spend hours trying to unpack it).
I expect that I should be able to tell from inspecting the memory dump of the running application (a ~300mb .DMP file) if a virtual machine is running or not. I just don't know what to look for. I've tried searching online, but annoyingly, queries for "VMProtect tell if virtualization is enabled" mostly return results about a feature of VMProtect which allows your protected application to tell if it is being run in a virtual machine (as in, for the purposes of reverse-engineering) so that it can refuse to run--this is not what I'm asking about, but it involves almost all the same key terms.