Calculation with not defined variables?

Question

I am new to reverse engineering and try to solve a crackme with radare2.

My problem is understanding of the following lines:

|           0x000007a9      c745b8000000.  mov dword [var_48h], 0
|           0x000007b0      c745bc000000.  mov dword [var_44h], 0
|           0x000007cd      8b45b8         mov eax, dword [var_48h]
|           0x000007d0      4898           cdqe
|           0x000007d2      0fb64405c0     movzx eax, byte [rbp + rax - 0x40]

From the bottom part of this code surrounding them:

|           0x0000076a      55             push rbp
|           0x0000076b      4889e5         mov rbp, rsp
|           0x0000076e      4883ec50       sub rsp, 0x50               
|           0x00000772      64488b042528.  mov rax, qword fs:[0x28]    ; [0x28:8]=0x19c0 ; '('
|           0x0000077b      488945f8       mov qword [canary], rax
|           0x0000077f      31c0           xor eax, eax
|           0x00000781      c645b700       mov byte [var_49h], 0
|           0x00000785      488d3d580100.  lea rdi, qword str.Enter_password: ; 0x8e4 ; "Enter password:" ; const char *s
|           0x0000078c      e87ffeffff     call sym.imp.puts           ; int puts(const char *s)
|           0x00000791      488d45c0       lea rax, qword [var_40h]
|           0x00000795      4889c6         mov rsi, rax
|           0x00000798      488d3d550100.  lea rdi, qword [0x000008f4] ; "%s" ; const char *format
|           0x0000079f      b800000000     mov eax, 0
|           0x000007a4      e897feffff     call sym.imp.__isoc99_scanf ; String! ; int scanf(const char *format)
|           0x000007a9      c745b8000000.  mov dword [var_48h], 0
|           0x000007b0      c745bc000000.  mov dword [var_44h], 0
|           0x000007cd      8b45b8         mov eax, dword [var_48h]
|           0x000007d0      4898           cdqe
|           0x000007d2      0fb64405c0     movzx eax, byte [rbp + rax - 0x40]
|           0x000007d7      84c0           test al, al

I understand the output and that there is the input wanted (scanf). After that var_48h, var_44h and eax are set to 0. The next lines i don't understand at all. Beside the cdqe instruction there is some calculation(?) with the variables rbp and rax (=var_49h?).

Maybe you can help me to understand that?

Thank you in advance!

Answer 1

Lets take a look at the specific piece of code you can't understand:

mov dword [var_48h], 0
mov dword [var_44h], 0
mov eax, dword [var_48h]
cdqe
movzx eax, byte [rbp + rax - 0x40]

And go over it a few lines at a time.

The first couple of lines are either initializing a local variable of size qword (8 bytes) to 0 by assigning one dword at a time, or two dword-sized variables, each of which to zero:

mov dword [var_48h], 0
mov dword [var_44h], 0

Converting this to C, this may look similar to:

long var_ab = 0;

or

int var_a, var_b = 0;

The third line is a little puzzling assuming the above assignment is to a single qword variable:

mov eax, dword [var_48h]

Regardless, this does not translate to an actual C statement, but suggests the variable (either as var_ab or var_b will be used in following statements.

Taking a look at the following two lines makes it all clearer:

cdqe
movzx eax, byte [rbp + rax - 0x40]

The first of the two, cdqe simply sign-extends eax into rax, effectively setting the higher half of rdx to zero too. the second line of the last snippet makes doing so clear, after we consider the limitations of the x86 assembly language.

Since eax is used as an index for a stack variable, it is required to be sign-extended to rax because of assembly limitations. executing byte [rbp +eax - 0x40] is impossible in x86 64bit assembly, as all registers in offset calculations are required to be of the same length.

This makes it clear that the variables are indeed two dword variables, and eax is sign extended solely to be used in the following mov instruction.

The assembly code above, then, is probably the result of the following C code:

char buf[...];
dword idx = 0;
if (buf[idx] == 0) {
    ...
}

Answer 2

CDQE Convert DoubleWord To QuadWord

it takes whatever is in eax sign extends it and puts it in Rax

so as long as eax holds anything less than 7fffffff and greater than 0
Rax will be 00000000whateverwashereineax
when eax holds anything from 0x80000000 to 0xffffffff
rax will become ffffffffwhateverwashereineax

in your case eax is 0 (see var_48) so Rax will be 0 after execution of cdqe

and then it accesses a single byte (first letter of your password) and checks if it is 0 (test al,al)