0

I'm starting to learn amd86_64 calling convention and how it calls params and clean the stack but the concept hasn't sunk in yet, so i would like to get a help reading the params as this might clear the confusion for me in the future. I used the same function on a different processor architecture before but i'm sure the signature has changed by the devs because I tried to call it same as before but it crashes every time, here's the old signature:

void(*DrawText)(int, float x, float y, int color, const char * text)

Here's the assembly for the function in amd86_64 starting from the caller function:

    push    rbp
    mov     rbp, rsp
    push    r15
    push    r14
    push    r13
    push    r12
    push    rbx
    sub     rsp, 178h
    lea     rdi, [rbp+var_108]
    mov     esi, 1
    xor     edx, edx
    call    sub_937750
    lea     rax, dword_1885C10
    vcvtsi2ss xmm0, xmm0, dword ptr [rax+54h]
    vmovss  [rbp+var_164], xmm0
    vcvtsi2ss xmm0, xmm0, dword ptr [rax+58h]
    vmovss  [rbp+var_168], xmm0
    lea     rsi, a30f_1
    vmovss  xmm0, dword ptr cs:qword_10799B8
    xor     edi, edi
    call    sub_A0E930
    lea     rcx, qword_17FAB50
    mov     rcx, [rcx]
    cmp     byte ptr [rcx+2E87h], 0
    jnz     loc_4994
    cmp     byte ptr [rcx+2E81h], 0
    jz      loc_4994
    mov     eax, eax
    vcvtsi2ss xmm0, xmm0, rax
    vmovss  xmm1, [rbp+var_168]
    vsubss  xmm0, xmm1, xmm0
    lea     rax, unk_1871D60
    vmovss  xmm2, dword ptr [rax]
    vxorps  xmm1, xmm1, xmm1
    vucomiss xmm2, xmm1
    jbe     loc_3E5E
    vmovss  xmm1, dword ptr cs:qword_10799B8
    vdivss  xmm1, xmm1, xmm2
    vmovss  [rbp+var_16C], xmm1
    vcvtss2sd xmm2, xmm2, xmm2
    lea     rdi, [rbp+var_108]
    lea     rdx, a62fFps
    mov     eax, 0FF0000FFh
    mov     ecx, 0FF00FFFFh
    mov     r14d, 0FF00FF00h
    vucomiss xmm1, dword ptr cs:qword_10799B8+4
    cmova   r14d, ecx
    vaddss  xmm1, xmm1, dword ptr cs:qword_10799B8+8
    vucomiss xmm1, dword ptr cs:qword_10799B8+0Ch
    cmova   r14d, eax
    vmovss  [rbp+var_168], xmm0
    vmovss  xmm1, [rbp+var_164]
    mov     esi, r14d
    mov     al, 3
    call    DrawText

Here's what I gathered so far (not sure if it's right though), 1st arg is a pointer lea rdi, [rbp+var_108] 2nd and 3rd are floats which is the positioning coordinates:

vmovss  xmm1, dword ptr cs:qword_10799B8
vdivss  xmm1, xmm1, xmm2
vmovss  [rbp+var_16C], xmm1
vcvtss2sd xmm2, xmm2, xmm2

4th and 6th are int's (hex code for rgb colors I assume) mov esi, r14d and mov ecx, 0FF00FFFFh the 5th arg is a char pointer lea rdx, a62fFps. The function doesn't return anything so i'm sure it's a void.

void(*DrawText)(int *, float x, float y, int color, int, const char * text)

I tried to call this way but it also crashes, please let me know if I got something wrong and explain stuff because i'm little newbee when it comes to assembly.

Note: This is not a PLEASE DO IT FOR ME question but rather explain how do I read the params properly, I don't mind posting the final result at the end of the answer though.

1

You need to read from the bottom up. I have no experience with the AMD64 calling convention but here's my take:

Wikipedia says:

The first six integer or pointer arguments are passed in registers RDI, RSI, RDX, RCX, R8, R9 (R10 is used as a static chain pointer in case of nested functions[19]:21), while XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6 and XMM7 are used for certain floating point arguments.

Judging from that I'd say the following mapping would be right then:

Argument #  | Register   | Desc
------------|------------|---------
1           | rdi        | int *
2           | xmm0       | x
3           | xmm1       | y
4           | rsi        | int color
5           | rdx        | unnamed int
6           | rcx        | const char *text

The following was my mistake as I misread the post, the old signature posted at the top agrees with me, the one at the bottom does not

but when I worked through your code reading bottom up it didn't fit:

lea     rdx, a62fFps

that loads the text to print into rdx but if above mapping would be right, rdx should be some unnamed int, and only the next argument would be the text.

Note that this may be my fault because I may have missed some nuance of AMD64.

End of mistake

Back to what does work:

The first argument, the pointer, should be in rdi. If you read upwards,

lea     rdi, [rbp+var_108]

is the first line to write to rdi, so that's your pointer (it fits the type). Floats seem to be passed via xmm0 and xmm1 so those will be x and y. The color is the 2nd non-float argument, which would be in rsi and reading the code we find:

mov     esi, r14d

and rsi contains esi, so that would be your color, only it's 32 bit in size. If you read the code, it also seems to fit the color type.

Then above problem appears where I was expecting another int but the next argument seems to be the text instead.

  • Not sure if xmm0 getting used there? But if so then the first param should be a float not an int because by convention the xmm number reflects it's position. – Cyb3r May 5 '18 at 22:42
  • From my understanding the arguments are separated by type first. If you have a function with (char,float,int,float), we have two pairs (char,int) and (float,float) which get assigned to (rdi,rsi) and (xmm0,xmm1) – Johann Aydinbas May 6 '18 at 21:29
  • Then how do I know in which order they are? I've watched one tutorial saying that if, lets say third param is the first float argument then it'll get stored in xmm2 and so on... That might be only for Windows ABI? – Cyb3r May 6 '18 at 21:34
  • According to this: people.freebsd.org/~obrien/amd64-elf-abi.pdf it's as I described. Basically it just uses the next free xmmN register. See page 17 at the bottom for the process behind register allocation. – Johann Aydinbas May 6 '18 at 21:47
  • Appreciate your input but my question still persist, how do I know the order of mixed up params (ints and floats)? Been researching from last night but couldn't find a proper explanation so far. – Cyb3r May 7 '18 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.