What is __sighandler_t struct and purpose of SIGTRAP signal handler

Question

I'm reversing an ELF x86 binary which apparently has anti-reversing/anti-debuging protections, and one of the first thing the ELF does is to call a sys_signal with a SIGTRAP value :

.text:08048063                 mov     eax, 48         ; sys_signal
.text:08048068                 mov     ebx, 5          ; SIGTRAP
.text:0804806D                 mov     ecx, offset sub_80480E2
.text:08048072                 int     80h             ; LINUX - sys_signal
.text:08048074                 jmp     short loc_8048077

I have three questions :

• First, am I right to think that the purpose of this is to create a handler for the SIGTRAP signal, probably in order to prevent any debuger use
• Second, if this is creating a SIGTRAP handler, I should find a __sighandler_t at sub_80480E2, according to this x86 syscall table. How can I setup IDA so that it recognizes it as a __sighandler_t struct?
• Third, I could not find any detailed information about this structure. What is its composition?

I found in signal.h those lines :

/* Type of a signal handler.  */
typedef void (*__sighandler_t) (int);

Is __sighandler_t only a ptr to a function?

Thanks ! Feel free to tell me if I'm not beeing clear or if I forgot a usefull information.

Answer 1

First, am I right to think that the purpose of this is to create a handler for the SIGTRAP signal, probably in order to prevent any debuger use

When a SIGTRAP is raised, normally the handler given in parameter of signal is called. If you have a debugger attached, this function will not get called. If your handler is never called, you can assume a debugger is attached.

Here is a simple example:

#include <signal.h>
#include <stdio.h>

void on_trap(int n)
{
        puts("on_trap");
}

void callme(void)
{
        puts("hello");
        __asm__ volatile("int $0x03");
        puts("bye");
}

int main(void)
{
        signal(SIGTRAP, on_trap);
        callme();
        return 0;
}

Normal run:

$ ./antidebug
hello
on_trap
bye

With gdb:

gdb -q antidebug
Reading symbols from antidebug...(no debugging symbols found)...done.
(gdb) r
Starting program: /tmp/antidebug
hello

Program received signal SIGTRAP, Trace/breakpoint trap.
0x0000000008001174 in callme ()
(gdb) c
Continuing.
bye

Second, if this is creating a SIGTRAP handler, I should find a __sighandler_t at sub_80480E2, according to this x86 syscall table. How can I setup IDA so that it recognizes it as a __sighandler_t struct?

sighandler_t is a typedef for a function pointer, not a structure. If you want to define a memory area as a structure, you can use the shortcut ALT+Q.

Third, I could not find any detailed information about this structure. What is its composition?

sub_80480E2 contains code, as mentioned this is not a structure but a function pointer. Imagine you have the syscall sigaction instead, you can simply check the man page to see the structure definition. But IDA Pro has most of structures definition from standard libraries.

Is __sighandler_t only a ptr to a function?

Yes.