I'm trying to find out what value a certain parameter has when it's called (I know "parameters" are gone after compiled, but you get the idea). The pseudo-C and assembly code are:
sub_171F4A0(ctx, aes_mode, KEY, &IV); // EVP_DecryptInit(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *cipher, const unsigned char *key, const unsigned char *iv)
.text:000000000040CEE6 ; 252: sub_171F4A0(v110, v125, ptr, &v130);
.text:000000000040CEE6
.text:000000000040CEE6 loc_40CEE6: ; CODE XREF: sub_40CAD0+363j
.text:000000000040CEE6 mov rdi, [rsp+400h+var_400]
.text:000000000040CEEA lea rcx, [rsp+400h+var_1C0]
.text:000000000040CEF2 mov rsi, [rcx-28h]
.text:000000000040CEF6 mov rdx, [rsp+400h+ptr]
.text:000000000040CEFE call sub_171F4A0
I have already figured out IV value, but now I need KEY, which is 16 bytes.
Then I ran gdb ./executable (Linux 64-bit ELF binary) and set a breakpoint at 0x40CEF6 and examined all the registers mentioned above (I have no idea which one holds the third parameter).
(gdb) break *0x40CEF6
Breakpoint 1 at 0x40cef6
(gdb) run
Starting program: /path/to/executable
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, 0x000000000040cef6 in ?? ()
(gdb) x/4x $rdi
0x29d72f0: 0x00000000 0x00000000 0x00000000 0x00000000
(gdb) x/4x $rcx
0x7fffffffe340: 0x00000000 0x00000000 0x00000000 0x00000000
(gdb) x/4x $rsi
0x1c11620: 0x000001a3 0x00000010 0x00000010 0x00000010
(gdb) x/4x $rdx
0x2a09690: 0x724a5bac 0xa90b86f0 0xff9d8546 0x9910582b
None of those values is the actual key, but $rdx seems to be the best candidate to hold the key value.
However, since it's a pointer, I thought I should examine 0x724a5baca90b86f0, just in case, which didn't work:
(gdb) x/4x 0x724a5baca90b86f0
0x724a5baca90b86f0: Cannot access memory at address 0x724a5baca90b86f0
