18

I am trying to scan all possible techniques to disrupt the usage of a debugger on a Unix platform (ie POSIX and a bit more).

I am thinking about techniques such as the PTRACE test at the beginning of a program (or at various points of its execution), the insertion of fake breakpoints (eg int3/0xcc x86 opcode) or time checks. But, also global strategies defined on the program to slow down the analysis of the program.

For now, all the techniques I found on Internet were easily worked around as soon as the anti-debug technique has been understood. So, I wonder if there are stronger ones.

Improve this question
3
  • "all the techniques I found on Internet were easily worked around as soon as the anti-debug technique has been understood" <- this is the case with most of them but not a reason to discount them just because of that.
    – Igor Skochinsky ♦
    Mar 22 '13 at 17:42
  • You may want to check [this][1] topic. [1]: reverseengineering.stackexchange.com/questions/1930/…
    – jvoisin
    Jun 20 '13 at 16:44
  • You can't. It is impossible for any software to detect if it runs on a real machine or a perfect emulation which can be analyzed from the outside. Don't try to do something mathematically impossible.
    – 12431234123412341234123
    Oct 12 '20 at 17:45
20

Here's some I've seen or heard about:

  • Stripping the section headers. A simple and fully legal action that stops GDB dead in its tracks. Does not work against some other debuggers (e.g. IDA). Can be done using the sstrip tool.

  • Using syscall function or direct syscalls instructions instead of calling specific functions like ptrace(). Can be defeated by setting the breakpoint on syscall function or just stepping through the file, but may be non-obvious if you don't know about it.

  • Performing anti-debugging actions before main(), e.g. in constructors of global objects or using __attribute__((constructor)). Since GDB usually sets initial breakpoint in main(), it takes care of the default situation. Workaround is simple: put breakpoint on the actual file's entrypoint (info file in GDB).

  • Sending itself debugging-related signals like SIGTRAP. (Note that this can be ignored with handle SIGTRAP nostop in GDB.)

  • Forking and tracing itself with ptrace.

  • Fake breakpoints insertion: Inserting int3/0xcc will force the debugger to stop on these bytes as they will be treated as software breakpoints. If they are numerous it might slow down considerably the analysis.

  • Breakpoint detection: I saw this technique in this paper, you can attach a function that will be triggered when a breakpoint is encoutered. This paper covers also some other tricks.

Improve this answer
5
  • Ok, this seems a reasonable list of techniques but nothing surprising til now. And, you are missing the breakpoints tricks (insertion of fake breakpoints to confuse the debugger when PTRACEd and checking the hash of the next code block to see if new software breakpoints have been inserted or not). But, would you know a paper describing all these techniques for Unix (I know some papers for MS-Windows but none for Unix systems).
    – perror
    Mar 22 '13 at 17:42
  • @Emmanuel: feel free to edit it and add more, that's why I made it a wiki. I'm not aware of any papers, but possibly you can find some more tricks in something like Phrack.
    – Igor Skochinsky ♦
    Mar 22 '13 at 17:46
  • My IDA died while trying to load an ARM binary that had its section headers stripped. So #1 may still be a useful trick (though I ultimately wrote a script to reconstitute the headers).
    – nneonneo
    Jun 9 '14 at 1:38
  • @nneonneo: if that happens with the current version, please send a bug report. thanks.
    – Igor Skochinsky ♦
    Jun 9 '14 at 21:09
  • The real fun is fork and use ptrace() on yourself as a form of IPC.
    – Joshua
    Sep 7 '18 at 4:02
9

My understanding of anti-debugging technology is that it is a game of cat and mouse (or cat and also cat). A technique works until it is understood by the opposing side and then it doesn't work any more. I think that ultimately the advantage is on the side of the debugger. Consider dynamic binary instrumentation or virtual machine systems. You can detect the presence of DBI or emulation by looking for cracks or faults in the emulation of the platform, but those are just errors in the emulation/translation software. If you had a "perfect" emulation system, then a debugged application could not know that it was being traced?

So I think the best you can do is little things that are easily worked around. The Shiva system still I think represents "okay" protection from debugging on Unix platforms, since it decrypts small portions of itself to run on demand and then re-encryptes them, as I recall.

Improve this answer
1
  • I would tag the Shiva system as an advanced packer but not really an anti-debug technique (even if its main effect is to render the usage of a debugger extremely tedious because your window in memory where you can effectively set breakpoints is extremely small compared to a "normal" program).
    – perror
    Mar 22 '13 at 17:37
1

They aren't any, do no try to do the mathematically impossible.

For now, all the techniques I found on Internet were easily worked around as soon as the anti-debug technique has been understood. So, I wonder if there are stronger ones.

Yes, because there is no way of detecting a debugger that can not be faked. Software can not detect if it runs in a perfect emulation or in the real world. And a emulator can be stopped, the software can be analyzed, variables can be changed, basically everything can be done what can be done in a debugger.

Lets say you want to detect if the parent process is a debugger. So you make a system call to get the parent PID? The debugger can intercept the system call and return any PID which does not have to be the real PID. You want to intercept every SIGTRAP so the debugger can't use it anymore? Well the debugger can just stop in this case and send the SIGTRAP also to your process. You want to measure the time when you send SIGTRAP to know if the the process stops for a short time by the debugger for sending SIGTRAP so you know when there is a debugger? The debugger can replace your calls to get the time and return a fake time. Lets say you run on a Processor that has a instruction that returns the time, so no function call is needed to get the time. Now you can know that the time you are getting is real? No, the debugger can replace this instruction with a SIGTRAP instruction and return any time he wants or in case such a instruction does not exist, run the Software in a emulator that can be programmed in any way. Everything you can come up with to detect a debugger or emulator can be faked by the environment and you have 0 change to detect it.

The only way to stop debugging is by not giving the software to the customers but keep it in your hands. Make a cloud service and run the software on your server. In this case the customer can not debug your program since he does not run it and has no control over it. Except the customer can access the server or the data somehow, but that is a different story.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.