What it takes to write a simple PE file packer from scratch?

Question

I was wondering to ask here what it takes to write a tiny simple packer for PE files? I know how a PE file looks like but not in accurate details (if necessary I'll learn as much as it takes) and I know C/C++ and assembly but not as much as a pro.

the whole idea is to get acquainted with their implementations for further things like getting the idea how they work and ... as you know it's just the start point and when you hit it, you'll have a long journey ahead, like protectors and VMs, likewise.

but for now I need to know even if in general terms, what a packer looks like.I already googled about the subject but they lack enough technical information. if it's possible, introduce any book that could provide enough information beside practical examples or a project whatsoever.

Answer 1

This is a topic that has a ton of depth depending on how crazy you want to go but I'll try to break it down into chunks as much as possible and you can look at each one as much as you want. I'll mention the theory behind the steps and some details about how to do them mechanically as well.

Generally the packer is going to do these things:

• First, you'll take some executable code and turn it into a datablob using a reversible algorithm. The code could be a function, or it could be a whole program. A function is easy, a whole program requires more work.
• You'll embed the datablob somewhere that the packer would get to it. You could embed it as a resource in the program, download it from the internet, or use a custom program to manually insert the payload into your packer's executable.
• At runtime, your packer will find the datablob, run the reverse algorithm and copy the result into an executable buffer (generally VirtualAlloc, VirtualProtect or one of the NTDLL functions that those functions call). If you're embedding the payload as a resource you'll use one of the standard APIs to retrieve it. If you've manually embedded the payload in the packer's executable you'll need to have some sort of signature method that he packer and look for. Generally you'll call GetModuleHandle to get the base address of yourself at runtime, and then you can parse your packer's PE file in memory to find the datablob yourself

So with that said since you just want to get an insight on how they work and what to look for the easiest thing I can think of to do to have you write that could would be:

1. Write a small bit of assembly to do something trivial (add 2 numbers, etc) and copy those bytes somewhere
2. Write a simple XOR obfuscator to take those bytes and transform them into something that isn't code
3. Use the resource APIs I linked to embed your obfuscated code as a file in your packer
4. At runtime have the packer grab the resource, de-XOR the bytes, copy them to an executable buffer and call the function.

For step 4, in order to write the function you'll need to be able to write assembly code that conforms to a standard calling convention. MSDN has an article on them here. Generally, you'll need to do something like use VirtualAlloc to create an executable buffer and use typedefs to treat that executable buffer like a function call. Typedefs will also let you specify the calling convention.

If you want to write and pack a single function (not a whole program... yet) that performs some more sophisticated functionality, then I'd recommend looking into position independent code. There are tons of examples on the internet for how to use PIC in the context of exploitation, but the technique applies here as well. Most examples will be in assembly, but you can write PIC code in C as well by writing a function that takes function pointers as parameters. You'll find the function address of the pointer using LoadLibrary, GetModuleHandle and GetProcAddress.

Now, the real fun is if you want to write a packer that takes an entire executable image (PE file) and does what I mentioned above. If that's the case, you'll still need to do the obfuscation and embedding the executable, but in order for the packer to work correctly you'll also have to know enough of the PE image to set up a "normal" execution environment for the packed program. If this is what you want to do, I would look at tutorials that go over manually loading DLLs in memory as they have to solve many of the same problems as your packer will. You can start with the reference on Reflective DLL Injection and go from there.

In very broad detail, when you compile a program there is information embedded in something called your Import Address Table that tells the operating system what DLLs your program needs to run and what functions from those DLLs you'll be using. As part of the OS standing up your execution environment it makes sure to load any necessary DLLs in your address space, and then goes through your program's PE file to populate the IAT at runtime with the correct function addresses for whatever APIs you're using.

So essentially, from a development perspective if you want to use code to create sockets, you'll include Winsock2.h and add Ws2_32.lib to your linker dependencies. When your program starts, the OS is going to see this information embedded in your PE file, and load Ws2_32.dll into your processes address space for you, go into your IAT and update the placeholder function pointer to point to the loaded WSASocket (for example). As a packer you'll need to do all of this for the target program in order to have a "normal" environment. Parsing the PE is tedious but not very hard and between the DLL injection references there should be plenty of information available to you on how to do this.

Once you have those steps down you can look at the crazier stuff that packers do like antidebugging, custom import tables, advanced techniques to embed the information, etc.