IDA Pro's Super-Complicated Function Pointer Definition

Question

I have recently been debugging a binary and at a point I started to decompile a function. One of the lines of the decompiled file is this:

v14 = (int (__cdecl *)(signed int))sub_8048FB6(1);

I have been told it is a function pointer but I have no clue to what function it is pointing to. I will appreciate if someone breaks this absolutely vague string to me into pieces with elaboration.

Answer 1

The variable being assigned to:

v14 = 

The type cast needed to convert the result of the subroutine to the type of v14:

(int (__cdecl *)(signed int))

The subroutine call, with one argument: 1:

sub_8048FB6(1);

The typecast is needed because hexrays did not figure out automatically what the return type of sub_8048FB6 is, so it probably defaulted to int, instead of the function pointer.

---

Now the type:

The outer brackets denote a type cast:

(int (__cdecl *)(signed int))
^                           ^

The calling convention cdecl is cpu specific, commonly, a couple of arguments in registers, the rest on the stack, with the last argument pushed first:

(int (__cdecl *)(signed int))
      ^^^^^^^

It is a function pointer, denoted by the bracketed (...*)

(int (__cdecl *)(signed int))
     ^        ^^

A function taking one argument, a signed integer:

(int (__cdecl *)(signed int))
                ^^^^^^^^^^^^

And the function returning an integer:

(int (__cdecl *)(signed int))
 ^^^

---

This is the same as you would declare a function pointer in C:

typedef  int (*myfunctype)(signed int);
int afunction(signed int arg);
myfunctype  fp = afunction;

---

If you want to know what function pointer it is that is returned, you will have to look inside sub_8048FB6, to see where it gets it’s return value from.

For example, sub_8048FB6 may something look like this:

(int (__cdecl *)(signed int)) sub_8048FB6(int a1)
{
     switch(a1) {
        case 1:
            return sub_80123456;
        case 2:
            return sub_80456789;
    }
}

And elsewhere, the returned functions:

int sub_80123456(signed int)
{
   …
}

int sub_80456789(signed int)
{
   …
}