Currently I'm trying to get past Enigma's loader, which asks you for a name & key, as an excercise.
I have very little progress, as I haven't worked too much with packers.
What I did was breakpoint on
CreateWindowExA, and then I put random data at key / name, and then click
Register, which effectively breaks at
From there, I'm able to check the stack and go back a few calls, but I'm really lost as there's a lot of bloat code (short functions with x87 instructions), and I don't know where to continue.
I also know that the routine that checks the serial is virtualized, so I should patch the places which check the return of the function that checks the serial instead of the function itself, but then again, I get lost when looking at the stack.
Here's my progress so far:
The breakpoint on
CreateWindowExA, which gets called when you input an invalid serial
The function that calls it: http://screenshotuploader.com/i/1502/t_q0c.png
And that function, before and after calling
CreateWindowExA, calls these two weird x87 functions:
the function that is marked in the previous screenshot
this one gets called a few instructions after
After going one more function back, I see:
The problem with that function is that it's using some tricks to change the return address (I guess), like this:
Which then leads to weird code which expects a function pointer in
edx returned by a funtion that is dereferencing a pointer to a
word (2 bytes)!
Here's the code that calls the function that returns a pointer to a function:
And here's the function that returns a pointer in
It is really weird, as it 1) derefs a
word (2 bytes, not enough for a function pointer!), and 2)
FLDCW loads a x87 FPU control word... what's happening here?!
Also, that's the last return I can see in the stack, as the rest of the values are zeroed, and there's no branch which checks the serial at this depth, so I don't know where to continue looking at.
And if not enough, the program closes each time I input an invalid serial, which makes it harder, because the next time the offsets are totally different and I can't find the code so I breakpoint a bit before, so I'm really lost!
Dumping the executable at runtime does absolutely nothing, as it's still packed as it was before, it just makes it 4x bigger (50MB -> 200MB), and doesn't help at all.
Any help would be great, even a simple idea, like a hint on what to do next, where to breakpoint, where to look at, or whatever you can think of, would help a lot.