In my BA thesis, I wanted to present a program for Windows, using various anti-debugging techniques.

One of them, which I came up on my own (possibly it is well known, but I didn't find it anywhere) involves changing control flow by using signal handlers. As far as I know, when program receives certain signal and has a function registered to handle it, the main thread is paused and the execution is passed to the handler. After it returns from the function, the main thread can either continue execution, or even terminate in case where SIGFPE or SIGSEGV was passed, for example (see this question).

However, in such a case, it is possible to cause the thread executing handler not to stop after it leaves handler scope. The anti-debugging technique I've mentioned before takes advantage of this fact and is presented below.

First of all, I'm aware that the code I'm about to present uses concepts that should never appear in a real program, should be avoided in almost every situation and never tried at home :). After establishing this, consider the following program: (I'm using GCC targeting minGW to compile it)

#include <csignal>
#include <windows.h>
#include <cstdio>

void handler(int)
    asm("jmp continue"); // do not return from this function; just jump to another place in the code

int main()
    signal(SIGFPE, handler); // I know that sigaction should be used instead, but it's simpler and, as far as I know it wouldn't work for Windows
    // and I know, that after receiving second signal, there will be undefined behaviour, but let's assume it won't happen
    int a = 1 / 0; // cause SIGFPE to happen
    printf("continue execution\n");
    Sleep(5000); // to show a printed message for a while, so that the console is not being closed immediately

So here comes my question: how could be such a program debugged, such that it's possible to see printf("continue execution\n"); line being executed?

When I tried to do this using Code::Blocks debugger (GDB), only the information that a program received SIGFPE appeared and the program crushed without printing "continue execution" string.

On the other hand, when I tried to debug it in OllyDbg, execution just stopped, nothing was printed on the screen, and I wasn't able to continue execution to the point where this string should be printed.

Upon "normal" execution it prints the string end finishes as expected.

Edit: I've also tried to pass the exception to the application (using SHIFT+F7), but I only get message saying "Debugged program was unable to process exception" and I'm unable to continue the execution.

Edit2: I will not upload the binary, since when I tried to download it, my antivirus software started to complain about it, so it would probably be marked as suspicious if someone else downloaded it.

  • signal handlers are not really a thing on Windows; are you using cygwin or mingw? – Igor Skochinsky May 27 '19 at 10:51
  • Well, I'm using Windows 10. – bart1e May 27 '19 at 10:52
  • what compiler are you using to build the program? – Igor Skochinsky May 27 '19 at 10:53
  • I'm using GCC. I'll update my question to contain this information. – bart1e May 27 '19 at 10:54
  • 1
    Targeting minGW. – bart1e May 27 '19 at 11:06

This technique is equivalent to the Structured Exception Handler for Windows. The application registers an exception handler, then forces an exception to occur, thereby transferring control to the exception handler. However, when a debugger is present, it will generally intercept the exception to allow user interaction. If your debugger allows you to pass the exception back to the program, then you will be able to continue tracing.

You can read more about the technique here: The "Ultimate" Anti-Debugging Reference (pdf)

  • Thank you for your answer and for posting a link to the great book. :) The thing is, I've already tried to pass the exception to the application by debugger (OllyDbg), but I just get message saying "Debugged program was unable to process exception" and the execution doesn't continue. I will update my question with a binary and with this information. – bart1e Jun 1 '19 at 8:49

Not sure if the exact answer you are looking for is in there, but this presentation is one of the best I have seen on this topic: https://www.blackhat.com/presentations/bh-usa-07/Yason/Presentation/bh-usa-07-yason.pdf

  • I didn't find the answer there, but it's good article nonetheless. Thank you for posting it. – bart1e May 27 '19 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.