How does the different types of executable packing work?

Question

Reference to this thread: What are the different types of packers?

Q1: How do packing methods like compression, protection, and encryption work, how do they differ, and which is more secure?

Q2: Is nested packing useful? i.e: packing via compression, and then protection or any combination of the referenced methods.

I am new to the RE world and am looking to prevent RE on an executable as best as I can. My thoughts are to obfuscate the source, build the executable, compress the executable (using a compressor), and then protect the executable (using a compressor).

I understand that RE is always possible with enough time by a skilled adversary.

Answer 1

Q1: I would refer you to the book Practical Malware Analysis Ch. 18 which talks about packers and unpacking them. You can find the book at any online book store and usually comes highly recommended for beginners.

Q2: I don't often see nested packers but they are out there most of the time when you see them nested the inner layers are for protection and the out layers are for compression (keeping the payload small).

Here are some links that may help you:

1. https://0x00sec.org/t/packers-executable-compression-and-data-obfuscation/847
2. https://www.nostarch.com/malware