I was wondering what are some advanced AntiDebugging techniques that more advanced than the basic ones like IsDebuggerPresent and CheckRemoteDebuggerPresent?
Similar to those 2 APIs, there are similar ways to check about the presence of debugger. For instance:
- Checking CPU Ticking
- Time it takes to complete a preknown action
- switching from 64 to 32 and vice versa if supported (WOW)
- Loaded libraries(similar to ASLR bypassing) that might indicate the presence of a debugger or some sort of VM
- Specific attacks to confuse the way a specific debugger or reversing tool interprets the data (For instance by abusing difference between sweeps, in IDA for instance linear Others)
- Writing a custom ASM that follows the program logic(meaning won't break the program), but will make the reversing tool give you incorrect information
Those are just glimpses of the almost infinite ways to trick the debugger. However, a good reverser will overcome all of this eventually, but sometimes the goal is not make it cost efficient or confuse the reverser so he won't decide to spend time on it by making him think the purpose is other or that the functionality is lacking and something is broken. There are a lot of reasons and a lot of ways to overcome them, and vice-versa.
Hope I could help a bit.