I got an assignment to analize an exe file with 97% entropy. It's obviously packed but I got no results from Protection Id or PEid about which packer it used...

How can I unpack it if it's possible? Or should I just throw it to ida and hope for the best? Everything I've found on the internet was either too complicated for me or used known packers. Sorry if this is an easy topic to understand, I'm new to this stuff. Every answer is appreciated!

  • 3
    The short answer is to start debugging, at some point there will be an unconditional jump to the unpacked code, dump the binary at that point and fix it up. There are many unpacking tutorials available out there, don't be afraid to try and fail, that's part of learning. Sep 29, 2022 at 23:11

1 Answer 1


You don't throw things at IDA and hope for the best and, as an analyst, you don't relay on automatic tools this much. That's clearly a sign you need to start doing the heavy lifting.

A packer is simply a binary, so you proceed as you would do with any other binary. There is no general procedure, I'll tell you what I usually do but I may have forgotten something and the list cannot possibly be exhaustive.

The general rule is: you debug it until you understand it, no matter how much time and focus will it take (5 min, 1 hour, 3 days, a month, doesn't matter).
You do this often enough and the time will shorten further and further until it's acceptable, you never get your hands dirty (i.e. use automated tools) and you'll never learn how to do this.

Anyway, I'd do this assuming the packer is a PE:

  1. Inspect the PE with a PE viewer. I use CFF Explorer. This will tell you if the PE is 32 or 64-bit, if it's a .NET assembly and what resources it has. Very simple packers have plain or XOR/RC4 encrypted payloads in their resources. The resources will also tell you if you are dealing with a Delphi binary, a particular runtime, an installer, UPX and so on. With experience, you can even tell malware and packer families aparts just by inspecting their PE.
  2. If it's a .NET use dnspy to inspect and debug it.
  3. If it's a Delphi use IDR to inspect and debug it.
  4. If it's a VB6 use VB decompiler.
  5. If it's AutoIT or similar scripted automation, use the relevant decompiler.
  6. If it's an installer (NSIS, MSI, SFX, ...) use 7z 9.38 to extract it and then go back to point 1 (you may find out the packer it's an installer later, it's OK, you are required to know how the main installer looks like when disassembled with IDA, Ghidra or similar).
  7. If it's not either use IDA.
  8. If necessary debug the packer (I use x64dbg).

.NET assembly

These can be easy to unpack or can be hard (if they patch metadata or use delegate shadowing).
In either case, check any module constructor and variable initializer. Often the malicious code is hidden inside a legal application, you have to find it starting from the entry-point. This is tedious but not hard, it's easy to spot malicious code (you are required to be a programmer to be a malware analyst, so you surely know what good code looks like).
Often the packers use Control-Flow Obfuscation, this makes the code hard to read but not too hard to debug.
What these packers usually do is decode an assembly (a DLL) from the resource and load it with Assembly.Load then use reflections (GetTypes, GetMethods, Invoke) to invoke a method. Use the module view of dnspy to put breakpoints in dynamically loaded assembly.
By debugging and inspecting the local variable you can easily find the configuration and the payload. You'll know when you are in the final unpacking routine because there will be a lot of checks on the configuration (cfr. MaaS).
Keep in mind that there can be one, two, five, or even ten additional assemblies before the final unpacking. If the packer uses "delegate shadowing" (i.e. call its method and runtime methods through delegates) you can inspect the target of the delegate in the local window to see if you are going to step into a runtime function.

De4dot was once useful for deobfuscating .NET assembly, not it's pretty useless but for dynamically executing string deobfuscation and renaming.

dnlib is a very useful library to manipulate IL code, you are required to know how to use it to write deobfuscator or simplify the analysis.


This is a standard Delphi executable, as a malware analyst you are required to be confident with the Delphi runtime (particularly string and array manipulation, Unit initialization and GUI events).
You may need to download additional "Knowledge Bases" to have IDR recognize the Delphi runtime.
Your first task is finding the malicious code. This can be in a unit initialization routine, in a timer in a form or in any form load/create event.
Use IDR to inspect the possible entry-point and look for anything suspicious, use the rename function copiously.
Look for the usual tricks (PEB accesses, LoadLibrary and so on). Generally speaking, the packer will allocate memory to decode the payload and then map and execute it.
There is no way to give a general procedure. Note that you want the payload just decoded but not yet mapped (the payload is usually a PE) otherwise you have to patch it once dumped.


First identify the runtime used, for example, Go, Rust, C, C++ or so on. You are required to know how to tell the main programming languages apart (including Delphi, sometimes it can be wrongly detected).
If the languages have metadata (e.g. Go, Rust) then use the necessary IDA script to annotate the DB from these metadata (e.g. Go has AlphaGolang). If the scripts don't exist, write them. As a malware analyst, you are required to be confident in the vast majority of programming languages and platforms, including idapy.
See also if FLIRT or Lumina can help you (particularly for C/C++).

Once you have made IDA detect as much library code as possible, you need to find and inspect the entry-point.
Get a gist of the possible obfuscation used and if used learn the patterns, what they actually mean and how to mentally translate them.
Learn the shape of the decoy code to ignore (this can be done with several years of experience in programming, decoy code makes no sense so it really stands out as you have never seen it while debugging the legal programs you worked on) and look for the suspicious one (again: PEB access, allocation of memory, loading of modules).
The packer will generally decode the payload in allocated memory and then map and execute it.
The memory can be allocated with a variety of functions (NtAllocateVirtualMemory, GlobalAlloc, LocalAlloc, VirtualAlloc, on a writable section, ...).
See if the decoding code is simple (short actually) enough to be worth emulating in a script or if it's easier to debug it. You can expect an intermediate shellcode, this can be called with anything (indirect jmp and call, push/ret pairs, callbacks in APIs, ...).
Again, you want the payload decoded but not mapped.


If the packer is really written in VB6, just read the decompiled code. Again it should be easy to spot the decoding routine thanks to the boatload of metadata.
Often the VB6 binary is just a decoy, you have to find the call to the shellcode. This is usually a simple call to a fixed address that is not listed as a VB6 routine. Put a breakpoint there and debug.

You are also required to know the anti-debug tricks and how to work around them (you can simply put breakpoint on APIs).

  • 1
    Thank you for taking your time to write such detailed answer, it's very helpful. I also use CFF Explorer and I know for sure it's not any of those you've listed above. Your IDA explanation is very clear and I'm sure I will come back here when I need help for the others. I'm very grateful, thank you!
    – beytrod
    Oct 1, 2022 at 0:05
  • 1
    @beytrod If you want (and can) to share the binary, I could take a look at it out of curiosity. Oct 1, 2022 at 11:47
  • Sorry, I cannot share it for 3 more weeks (until the assignment period ends). I will share it if you're still interested by then.
    – beytrod
    Oct 2, 2022 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.