I have a stripped binary, which I want to reverse engineer. I spent quite some time with it, and dug myself through most of it, however, I got stuck near these lines. Context: the program has 5 local int64 variables (s, v2, v3, v4, v5 respectively) and an int (v6) next to each other in the memory, and these get initialized to 0LL immediately.
char s[8]; // [rsp+B0h] [rbp-50h]
__int64 v2; // [rsp+B8h] [rbp-48h]
__int64 v3; // [rsp+C0h] [rbp-40h]
__int64 v4; // [rsp+C8h] [rbp-38h]
__int64 v5; // [rsp+D0h] [rbp-30h]
int v6; // [rsp+D8h] [rbp-28h]
It reads 45 bytes from its first argument to the the address where s begin, therefore it fills all the variables with bytes. Then after numerous if checks and arithmetics I have managed to figure out how the first 21 bytes should look like. Now here comes the critical lines (disassembled by IDA):
xorEachK((__int64)&unk_40A0, int_xor_range1, SBYTE6(v3));
v10 = (unsigned int (__fastcall *)(char *, signed __int64))&unk_40A0;
addr1 = (void *)((unsigned __int64)&unk_40A0 & 0xFFFFFFFFFFFFF001LL);
mprotect((void *)((unsigned __int64)&unk_40A0 & 0xFFFFFFFFFFFFF001LL), 4096uLL, 7); //rwx
if ( v10((char *)&v3 + 7, 4096LL) == 1 )
{
xorEachK((__int64)&unk_4100, int_xor_range2, SBYTE4(v5));
v12 = (unsigned int (__fastcall *)(char *, signed __int64))&unk_4100;
addr2 = (void *)((unsigned __int64)&unk_4100 & 0xFFFFFFFFFFFFF001LL);
mprotect((void *)((unsigned __int64)&unk_4100 & 0xFFFFFFFFFFFFF001LL), 4096uLL, 7);// rwx
if ( v12((char *)&v5 + 5, 4096LL) == 1 )
{
printf("%s\n", s, argv);
result = 0LL;
}
} else {/*show error, return ...*/}
unk_40a0, and unk_4100 are global variables (with unknown type), their values are:
.data:0000000000004100 unk_4100 db 21h ;
.data:00000000000040A0 unk_40A0 db 97h ;
They are on top of eachother in the data segment.
xorEachK function is simple as that:
void xorEachK(__int64 a1, signed int a2, char a3)
{
for ( signed int i = 0; i < a2; ++i )
*(_BYTE *)(i + a1) ^= a3;
}
int_xor_range1, 2 are also global variables, their values are: 92, 493
v10, v12 seem to be some kind of (virtual??) function pointer:
unsigned int (__fastcall *v10)(char *, signed __int64); // [rsp+30h] [rbp-D0h]
unsigned int (__fastcall *v12)(char *, signed __int64); // [rsp+40h] [rbp-C0h]
And I really fail to realize what is going on near the mprotect. I do understand what mprotect does, it grants rwx permissions to the memory from addr1, and 2 for 4096 bytes, but I dont understand the other lines especially the if(v10(...) ==1) type of calls. What are v10 and 12?
Also, am I right thinking that, the address is created by the bitwise and of 0xFFFFFFFFFFFFF001LL and the address of the unknown types? 0xFFF.. seems such a huge address it is not even in the stack.
Eventually, I also need to figure out how these lines contribute to the values of s and the rest of the local variables. And since printf prints out bytes until the terminating '\0' I only need to figure out which are the bytes of s should not be 0, since they are initialized to 0 at the beginning.
v10
andv12
are encrypted functions which are being decrypted withxorEachK
. To see their decrypted form you have to either put breakpoints just after calls toxorEachK
or decrypt them statically if you know values ofa1
,a2
anda3
. – bart1e Mar 15 at 19:19