Reverse Engineering Stack Exchange is a question and answer site for researchers and developers who explore the principles of a system through analysis of its structure, function, and operation. It only takes a minute to sign up.
Sign up to join this community
i think its pretty easy to find the OEP when the packer runs from a different section. there are many tricks, such as running a trace until you hit the codesection, or breaking on writes to the IAT, and putting a bp on the codesection after the last write to the IAT.
but what about packer code that runs in the original code codesection? what are some tips and tricks to find the OEP, since the packer is running from the same section,and jumps to the OEP in the same section?
Generally look for register clearing (pushad/popad) followed by a jump. Using something like mona.py in immunity you could scan the process's address space for a sequence of instructions similar to that. Keep in mind that these things are packer specific, so it might not work for all of them.
You could also use some kind of DBI or memory monitoring tool to break & dump when that section's memory stops being modified (when the program finishes unpacking). I've done something similar using Intel's PIN before.
You could place a breakpoint on LoadLibrary and GetProcAddress API functions. This is a nice way of doing this because most packers use these functions to load the needed libraries and to fix the import table. Based on my personal experience import fixing is mostly done at the very end of the unpacking process.
After you are sure that the imports are fixed, it is possible that there will be a small decryption routine which will decrypt the OEP and make a jump to it. But, as mentioned in the answer above, this is packer specific and it might not work for all of them.
