12

I have some code (I assume Delphi) which uses only the EAX and EDX register for passing the arguments (and of course the stack if more are required). I looked which calling conventions would match, but I haven't found one which uses only EAX and EDX. AFAIK Borland fastcall/register is using EAX and EDX, but also ECX, which is not the case here.

Can I tell IDA somehow about this calling convention? How would I do this?

  • Can you paste some code which uses EAX, EDX and stack without ECX? – Igor Skochinsky Oct 28 '13 at 11:21
  • After looking into this in more detail, it seems that ECX is indeed used, and that the code usually doesn't need more than two arguments. When I looked at functions with more arguments, I realized that it was the standard with EAX/EDX/ECX/PUSH... – Devolus Oct 28 '13 at 12:14
  • Okay, so mystery solved then. – Igor Skochinsky Oct 28 '13 at 12:22
21

you can add a function type by editing it (Key Y) and adding the name.

I wrote a page to remind me about calling conventions at ASM level.

Introduction

the original call is myfunc(0,1,2,3,4).

  • standard order is first argument is pushed last.
  • standard stack adjusting is 'callee cleanup' - after returning, the stack should be without its calling arguments.

Note: the stack looks vertically like the call order.

stdcall (stack only)

push    4
push    3
push    2
push    1
push    0
call    myfunc
xor     eax,eax
retn    10

Fastcall (ecx, edx)

This is actually Microsoft's fastcall.

push    4
push    3
push    2
mov     edx,1
xor     ecx,ecx
call    myfunc
xor     eax,eax
retn    10

CDECL & syscall (caller cleanup)

push    4
push    3
push    2
push    1
push    0
call    myfunc
add     esp,014
xor     eax,eax
retn    10

Pascal (reverse order, ebx saved, even if ebx is unused...)

push    ebx
push    0
push    1
push    2
push    3
push    4
call    myfunc
xor     eax,eax
pop     ebx
retn    10

Fortran/watcall (eax, edx, ebx, ecx, then stack - ebx is saved)

Apparently it's not so clear what the fortran calling convention is, and this one is even different from raymond's post's The __fortran calling convention isn't the calling convention used by FORTRAN.

push    ebx
push    4
mov     ecx,3
mov     ebx,2
mov     edx,1
xor     eax,eax
call    myfunc
xor     eax,eax
pop     ebx
retn    10

Delphi 'registers' calling convention (default)

Uses eax, ecx, edx as first 3 arguments. Other arguments are pushed on stack in reverse order.

push    3
push    4
mov     ecx, 2
mov     edx, 1
xor     eax,eax
call    myfunc
xor     eax,eax
retn    10
  • The retn 10 line at the end of each example is a red herring and likely to confuse. Its value has nothing to do with the examples' calling of myfunc. – usr2564301 Jan 20 '17 at 11:17
10

If you run into a calling convention which is not covered by any of the standard calling conventions you can use the __usercall or __userpurge calling convention which allows you to specify which arguments are passed where. The syntax is

return_type __usercall function_name<registers>(arg0_type arg0<registers>, arg1_type arg1<registers>, ...)

Where registers can be a grouping of registers separated by the ':' character if the argument occupies more than one register.

If you're in a really shitty spot calling convention wise you can use the full syntax as described here. Where you instead of simply typing the register name holding the argument you can describe arguments being passed in parts of registers or stack elements. The syntax is as above only with registers replaced by

<argoff:register^regoff.size>

or if the argument is passed on the stack

<argoff:^stkoff.size>
5

Delphi and Borland C++ Builder use EAX, EDX and ECX for the first three arguments in their variant of the __fastcall calling convention. So if you choose "Delphi" or "C++ Builder" in Options-Compiler, you can just use __fastcall in the function prototype - no need to resort to __usercall.

  • The original poster said that the calling convention only uses EAX and EDX not ECX. It sounds odd to me. – Peter Andersson Oct 28 '13 at 12:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.