I have a problem where I'm instrumenting stripped binaries; I don't know the start of main(). But there's always an init(), and init() calls libc_start_main(), which receives a pointer to main.

If I can instrument libc with analysis code to intercept the argument, then I can retrieve that address and place another pin callback there so that I can get it's arguments. The problem is, I don't know what the calling convention is; I was thinking, if I could boil this down to a matter of the calling convention, then I do this for any function. I did notice that gdb knows the calling convention of libc_start_main(), in fact it is so good, it knows the order of the arguments as well.

I did read a short note on stackoverflow that stated that the name of the function would yield the calling convention: « How to find the calling convention of a third party dll? »

If it's not possible to know the calling convention programmatically, what is the opinion on creating a local build of libc in order to be able to force a particular calling convention onto __libc_start_main()... you see my chain of thought. Does anybody think that this is a better approach, rather than solving it in the general case ?

  • I am afraid that solving the general case would require to follow the whole initialization function and track the final jump to the main() function. Meaning that it is probably much more efficient to have a collection of standard initialization functions than trying to solve it in the general case. – perror Sep 22 '14 at 14:07

Since you mention __libc_start_main it seems you need to worry only about Linux. In such case, the calling convention is known and fixed. In most cases it will be SystemV ABI or similar.

| improve this answer | |

I found that (at least on Linux) it was reliable to intercept _init and to read from a constant offset from the stack pointer was pretty reliable. I ended up producing a pintool that would do just that.

| improve this answer | |
  • Well, your question was a classical XY problem: you needed to solve a specific, narrow case but you stated it in a very generic way. – Igor Skochinsky Sep 24 '14 at 23:11
  • I agree that your response is a correct/sufficient solution to the problem posed. – Adam Miller Oct 3 '14 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.