I have an dll that is loaded into a 64 bit process, it performs two check sums (at least) one to keep up to date (it receives a 13 character long string from the net which I'm fairly certain is an in house hash as it never changes and doesn't seem to corespond to any hash pattern I can think of 11 numbers 2 uppercase letters at the end) and the other to defeat people trying to patch/analyze behaviour of different execution paths, it will refuse to run if it's checksum is different than the expected one. I tested this by hex editing a string the program uses which resulted in the dll never running, patching it during runtime with a memory editor did work. I guess the real question is how would I go about identifying where the checksumming happens, what are the common imports I should be hunting using IDA etc.
1 Answer
It really depends on the program. You mentioned that it receives a saved checksum from the net and checks it against the computed value - one way you could do this is to look for recv() calls (or whatever function they use to handle incoming packets).
I will say that it would probably be much quicker to do this dynamically rather than statically. If you set a breakpoint on the packet handling function, you should be able to find where the check is occurring relatively easily (it should occur very soon after the 13 byte string is received). From there, you can work backwards and see where the computed value came from - if it's a pointer to the value, you can set a memory access breakpoint and run it again; if it's just the value, you can see how it was pushed onto the stack/accessed.
If you're dead set on doing it statically, good luck. Because it doesn't change from run to run, it's probably some kind of binary operation on the program - either a CRC or weak hash would be my guess.
