Hooking to a function with variable argument list

Question

I am using Detours 3.0 to hook onto a function I obtained from Ida Pro. The problem is I came across a function which I cannot seem to hook which is due to either invalid arguements or the fact that it uses a variable arguement list.

int (*MakePacketBuffer)(const char * buf, int len, const char * splitstr, ...);
#define MakePacketBufferProto int (*)(const char *, int, const char *, ...)

That is what I imagine the parameters to be like, in Ida Pro its actually shown like this:

int sub_66EEF0(int a1, int a2, const char *a3, ...)

How that function is called in Ida:

sub_66EEF0(buf, 2048, "%c%c%c%c%s%c%c", v15, 250, v20, v21, v22, v23, v25);

Answer 1

You'll need to know how the compiler generated the function. For 32-bit Windows applications compiled with MSVC, your life gets a little bit easier since all arguments are passed on the stack.

The way that the stack frame is set up upon entry to a routine on x86 Windows:

----------------
 Return Address 
----------------
 Argument #0 
----------------
 Argument #1
----------------
 Argument #2 
----------------
  ....

This continues to be the case for vararg routines. Here's a quick example that shows you how to get at the additional values. The issue that you'll run into if you're intercepting things like printf is that you have to parse the format string to know how many arguments have been passed to you.

#include <stdio.h>
#include <windows.h>
#include <intrin.h>

#define print(x) printf("%10s: %x\n", #x, x)

void fn(int a, int b, ...) {
    void ** Stack = (void**) _AddressOfReturnAddress();

    print(Stack);    // Address of SP upon function entry
    print(Stack[0]); // Return address
    print(Stack[1]); // a
    print(Stack[2]); // b
    print(Stack[3]); // vararg 1
};

void main() {
    fn(1,2,3,4,5,6);
}

This prints out:

   Stack: 101feac
Stack[0]: ce10a4
Stack[1]: 1
Stack[2]: 2
Stack[3]: 3

Answer 2

It is not necessary to know the number of arguments.

In practice, the number of arguments is limited by a constant (otherwise the program would need infinite memory :).

Therefore, you can declare a hook function with, say, 16 args, print the args, and call the original function with the same 16 args. If the code breaks, double the number of args.

typedef unsigned long DW;
#define DW_MANYARGS  DW arg1,DW arg2,DW arg3,DW arg4,DW arg5,DW arg6,DW arg7,DW arg8
#define MANYARGS     arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8

DW hookFunc(DW_MANYARGS) {
    // we do not expect a quadruple-word result here
    DW res = originalFunc(MANYARGS);
    return res;
}

Caveats:

Do not forget to return what the original function returns.

Remember that there are two hidden arguments:

0) the address of the return value, if the return value does not fit into the size of int;

1) the this pointer