I have wanted to get into the art of reverse engineering for quite some time now, so I took a look at a few online lessons (such as opensecuritytraining.info) and also got my hands on IDA Pro.
Obviously, since this is a complex topic, I was overwhelmed by registers, pointers, instructions et cetera. I know assembler and C fairly well, it's just (like I said earlier) a topic where you have to learn much.
Now to my actual question: I have downloaded a "CrackMe"-Program and started debugging it. Basically the objective is to find a key which you then have to enter into a Textbox in the program. I found the key checking function fairly easily and identified some logic, but I can't wrap my head around how the program actually compares the string. My C skills are much greater than my Assembler skills, so I decided to get some Pseudocode printed. The problem is that the Pseudocode is pretty messy (I'm guessing that's because of compiler optimizations) and I basically can't understand what this piece of code is supposed to do.
Here is the code (sorry it's a bit long):
int __usercall TSDIAppForm_Button1Click<eax>(int a1<eax>, int a2<ebx>, int a3<edi>, int a4<esi>)
{
int v4; // ebx@1
int v5; // esi@1
int v6; // eax@1
signed int v7; // eax@3
signed int v8; // edx@3
int v9; // ebx@3
int v11; // edx@12
int v12; // [sp-24h] [bp-34h]@1
int (*v13)(); // [sp-20h] [bp-30h]@1
int *v14; // [sp-1Ch] [bp-2Ch]@1
int v15; // [sp-18h] [bp-28h]@1
int (*v16)(); // [sp-14h] [bp-24h]@1
int *v17; // [sp-10h] [bp-20h]@1
int v18; // [sp-Ch] [bp-1Ch]@1
int v19; // [sp-8h] [bp-18h]@1
int v20; // [sp-4h] [bp-14h]@1
int v21; // [sp+0h] [bp-10h]@1
int v22; // [sp+4h] [bp-Ch]@1
int v23; // [sp+8h] [bp-8h]@2
void (__fastcall *v24)(int); // [sp+Ch] [bp-4h]@7
int v25; // [sp+10h] [bp+0h]@1
v22 = 0;
v21 = 0;
v20 = a2;
v19 = a4;
v18 = a3;
v5 = a1;
v17 = &v25;
v16 = loc_45C6FD;
v15 = *MK_FP(__FS__, 0);
*MK_FP(__FS__, 0) = &v15;
JUMPOUT(Controls__TControl__GetTextLen(*(_DWORD *)(a1 + 872)), 0xFu, *(unsigned int *)"j");
v6 = Controls__TControl__GetTextLen(*(_DWORD *)(a1 + 872));
System____linkproc___DynArraySetLength(v6);
System____linkproc___DynArraySetLength(664);
v14 = &v25;
v13 = loc_45C699;
v12 = *MK_FP(__FS__, 0);
*MK_FP(__FS__, 0) = &v12;
v4 = 0;
do
{
Controls__TControl__GetText(*(_DWORD *)(v5 + 872), &v21, v12);
*(_DWORD *)(v23 + 4 * v4) = *(_BYTE *)(v21 + v4 - 1);
++v4;
}
while ( v4 != 16 );
v8 = 1;
v9 = 0;
v7 = 4585384;
do
{
if ( v8 == 16 )
v8 = 1;
*(_BYTE *)(v22 + v9++) = *(_BYTE *)v7++ ^ *(_BYTE *)(v23 + 4 * v8++);
}
while ( v9 != 665 );
v24 = (void (__fastcall *)(int))v22;
// I know the key to success lies here, but I can't figure out what this if is supposed to do
if ( *(_BYTE *)v22 != 96 || *(_BYTE *)(v22 + 4) != 208 || *(_BYTE *)(v22 + 9) )
MessageBoxA_0(0, "Invalid Key", "Error", 0);
else
v24(v22);
*MK_FP(__FS__, 0) = v12;
v11 = v15;
*MK_FP(__FS__, 0) = v15;
System____linkproc___LStrClr(&v21, v11, 4572932);
System____linkproc___DynArrayClear(&v22, off_45C568);
return System____linkproc___DynArrayClear(&v23, off_45C548);
}
I would appreciate it if someone could give me advice on how to "deobfuscate" this piece of code. Are there any plugins available to do this? Is there a special technique that I can use to figure it out? Would it be better to look at the Assembler code instead of the Pseudocode?
I especially wonder where these weird constants (like 872 for example) come from.
Answers would be highly appreciated.