3

I'm writing a script that, among other things, reads and does some processing with parameters to a function. Right now my solution to find the number of arguments for a given function at addr is:

(GetStrucSize(GetFrame(addr)) - GetFrameSize(addr))/4

This is on x86. The issue is when I'm calling, for example, printf(), the difference between the two GetXXX() calls is 5 which obviously messes up the math.

At the moment I've worked around it by checking if the first parameters is a format string with format specifiers and then parsing it manually to get the real count, but that's not really extensible.

Is there a way in the confines of the API to deal with this?

Edit: I should have made this clearer in the question but this is specifically referring to the IDA API. I could write the code myself to calculate cleanup but in interest of keeping my code clean I'm trying to avoid a lot of code querying if it's stdcall and cdecl and then searching/parsing myself

1 Answer 1

4

For cdecl the caller needs to clean up the stack. So somewhere after the function call there HAS to be a stack operation that removes the parameters from the stack (remove as in adjust the stack pointer)

PUSH 0xABCDEF01
CALL function
ADD ESP, 4 ; sizeof(DWORD) (the parameter)

It does NOT need to be directly after the call. (I think VS08 sometimes puts the return value check before it) So this is equally valid

PUSH 0xABCDEF01
CALL function
TEST EAX, EAX
ADD ESP, 4 ; sizeof(DWORD) (the parameter)
; A conditional jump probably

as is this

PUSH 0xABCDEF01
CALL function
POP EAX 

Because EAX holds a DWORD and thus removes sizeof(DWORD) == 4 bytes from the stack. (This obviously also holds true for all other general purpose extended registers). (It is POSSIBLE that e.g. for N parameters one could use N POP but I'm certain that no compiler does that and for every N > 2 it will lead to bigger code so it is unrealistic to happen in handwritten assembly too.)

There doesn't exists a maximum N of instructions that you could check after the call to determine whether that stack operation is the cleanup code. However I think that 5 will work for almost all cases (If you by chance do already know that this function is CDECL then check as many as you want)

Also for functions where the return value isn't important but is called before the return value of another function is inspected, one can save a variable like this:

CALL function1
PUSH EAX ; Save EAX
PUSH something
CALL function2
POP EAX ; Remove the something parameter
POP EAX ; Restore the return value of function1

You could try to also count the PUSH but that comes with it's own sort of intermediate instructions - this is really common:

PUSH something
LEA EAX, something_else ; Get a pointer to something_else
PUSH EAX 
PUSH more
CALL function

So you cannot stop working 'up' as soon as you hit an instruction that isn't a PUSH.

In conclusion there isn't a perfect solution but I think both can lead to a pretty reliable value; Which one you choose (or maybe both?) is up to you, IMO the first one (Searching for the stack cleanup) is easier to implement.

5
  • Thanks for the answer; I'll probably accept this one as a solution. I understood cdecl's calling convention but was hoping there was an already implemented way to do this :(
    – Fewmitz
    Sep 4, 2014 at 18:08
  • @Fewmitz I'm sure someone can provide the proper name for those techniques which will probably lead to a finished implementation. I imagine it's a rather common task so someone's bound to have done it already
    – user45891
    Sep 4, 2014 at 18:11
  • Yeah I should have made it clearer that this is about the API specifically; I could write it myself but trying to keep the code clean, particularly when going to different architectures, etc.
    – Fewmitz
    Sep 4, 2014 at 18:15
  • 2
    Trivia factoid, I've seen a compiler optimize consecutive esp cleanups - when two cdecl functions were called in a sequence, the compiler (VC6) did not perform cleanup after the first call, only after the second one.
    – DCoder
    Sep 4, 2014 at 19:12
  • 1
    @DCoder Nice. I did a quick test and VS2008 still does that when any kind (speed or size) of optimization is enabled (With enabled link-time code-generation - I'm unsure whether module in that case also means printf()).
    – user45891
    Sep 4, 2014 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.