I am working on a crackme and I'm stuck on a particular comparison that I'd like your help in getting past(one of many). Here's what I know and the relevant code snippet will follow:

  • Requires a 19 digit number in the form xxxx-xxxx-xxxx-xxxx (no non-numeric characters). These are previous checks.

  • Looks like the 19 digits are added together, with 10 added on, divided by 4 (shifted right 2), and compared to some value on the stack (which I'm unable to determine).

Can you help me bypass the CMP on the last line ?

EDIT: Not allowed to NOP any instructions, need to find the serial number.

CODE: R9 is the user-inputted 19 digit string Added entire function

var_18          = byte ptr -18h
var_8           = qword ptr -8

sub     rsp, 18h        ; Follow four lines check for 19-digit user entry
cmp     edx, 19
mov     r8, rcx         ; Now holds User inputted string
jz      short loc_140001013

xor     eax, eax
add     rsp, 24

loc_140001013:                          ; CODE XREF: sub_140001000+Aj
xor     edx, edx
lea     rax, [r8+4]
mov     ecx, edx
xchg    ax, ax
db      66h, 66h
hyphen_Check:                           ; CODE XREF: sub_140001000+2Fj
cmp     byte ptr [rax], 2Dh ; Use a FOR loop to ensure - is every 5th character, repeats 3 times
jnz     short loc_14000100C
add     ecx, 1
add     rax, 5
cmp     ecx, 3
jb      short hyphen_Check

loc_140001031:                          ; DATA XREF: .rdata:00000001400020C4o
 ; .rdata:00000001400020D8o ...
mov     [rsp+18h+var_8], rbx
mov     r10d, edx
mov     r11d, edx
lea     rbx, [rsp+18h+var_18]
mov     r9, r8
db      66h, 66h
xchg    ax, ax
db      66h, 66h
xchg    ax, ax
db      66h, 66h
xchg    ax, ax
forLoopCheck:                           ; CODE XREF: sub_140001000+A2j
mov     rcx, rdx
movsx   eax, byte ptr [r9+rcx] ; Begin FOR loop to check all digits for non-numeric characters
add     eax, 0FFFFFFD0h
cmp     eax, 9          ; Non-number entries are checked here, exits if letters are present
ja      ExitFUNC
add     rcx, 1
cmp     rcx, 4
jl      short non_Numeric_Check ; END FOR LOOP


movsx   eax, byte ptr [r9+2] ; Get 3rd digit
                             ; Likely begins set of first checks on actual digits
movsx   ecx, byte ptr [r9+3] ; Get 4th digit
add     r11d, 1         ; Loop counter, loops 4 times, for each set of digits
add     ecx, eax        ; Set ECX to 3rd digit + 4th digit
movsx   eax, byte ptr [r9+1] ; Set EAX to 2nd digit
add     rbx, 4
add     ecx, eax        ; Adds 2nd digit to total
movsx   eax, byte ptr [r9] ; Move first digit into EAX
add     r9, 5           ; Moves to 2nd set of digits
add     ecx, eax        ; Adds 1st digit to total
add     ecx, r11d       ; Adds 1, for a total of 10
mov     [rbx-4], ecx    ; Added four digits stored here too
add     r10d, ecx       ; R10d becomes our added ECX value from above
cmp     r11d, 4         ; Loop counter check
jb      short forLoopCheck
shr     r10d, 2         ; shifts right r10d by 2, AKA divides by 4
mov     ecx, edx
lea     rax, [rsp+18h+var_18] ; var_18 = -18h
xchg    ax, ax          ; A NO OP
cmp     [rax], r10d     ; Compares right shifted 2 value to whatever is in RAX
jnz     short ExitFUNC  ; Compare above must be the same value


.text:00000001400010C4 loc_1400010C4:                          ; CODE XREF: sub_140001000+ECj
.text:00000001400010C4                 movzx   ecx, byte ptr [r8+rax+15] ; Move last set of 4 into ECX
.text:00000001400010CA                 cmp     [r8+rax], cl    ; CL = 1st digit of last 4, R8 = 1st dig of 1 set
.text:00000001400010CE                 jz      short ExitFUNC  ; Kicks you out
.text:00000001400010D0                 movzx   r9d, byte ptr [rax+r8+5]
.text:00000001400010D6                 cmp     cl, r9b         ; cmp 1st digit of last 4 with 1st digit of 2nd 4
.text:00000001400010D9                 jz      short ExitFUNC
.text:00000001400010DB                 cmp     r9b, [rax+r8+10] ; cmp 1st digit of 2nd set to 1st digit of 3rd set
.text:00000001400010E0                 jz      short ExitFUNC
.text:00000001400010E2                 add     edx, 1
.text:00000001400010E5                 add     rax, 1
.text:00000001400010E9                 cmp     edx, 4
.text:00000001400010EC                 jb      short loc_1400010C4
  • nop the jnz and done?
    – rev
    Aug 22, 2015 at 19:05
  • I'm required to figure out a certain serial number, so NOPing the jumps is out of the question.
    – Asa Hunt
    Aug 22, 2015 at 19:18
  • well, you said 'bypass'... anyway, can you share the binary you're reversing, or it's private?
    – rev
    Aug 22, 2015 at 19:33
  • I wouldn't want to share it publicly, but I could send it to you in private. Not sure how that would work though.
    – Asa Hunt
    Aug 22, 2015 at 19:46
  • Please add the forLoopCheck label to your assembly, at the moment, i'll assume it's right at the top. If you don't want to nop out the cmp, you'll need to find out what it compares with; set a breakpoint and run if neccesary. From this code, it seems any 16 digits that sum up to the correct value should work, but since the individual sums are stored at the [rbx] array as well, be prepared that more checks are likely to be done with those values. Aug 22, 2015 at 19:59

1 Answer 1


Here's what your serial checking algorithm looks like:

// expects xxxx-xxxx-xxxx-xxxx
bool check_serial(char inp[]){
    int total = 0;
    int checksum = 1 + (inp[0] + inp[1] + inp[2] + inp[3]);

    for (int i = 1; i <= 4; i++) {
        total += i + (inp[0] + inp[1] + inp[2] + inp[3]);
        inp += 5;

    total /= 4;

    printf("chk is %d, total is %d\n", checksum, total);

    return (checksum == total);

I'll explain what the value at rsp is only, as you seem to understand the rest of the algorithm.

First, you have this instruction:

cmp     [rax], r10d

Check what rax is:

lea     rax, [rsp+18h+var_18]

Since var_18 is -0x18, we know that 0x18 + (-0x18) is 0, so the above is equivalent to:

lea     rax, [rsp]

(you can put the cursor over 18h and press K in IDA, it'll get rid of the variable name and just show [rsp])

Which is equivalent to:

mov     rax, rsp

Which means that your cmp is doing:

cmp     [rsp], r10d

Now, you need to find out what's on the stack. For that, you check the function again, and search for any references to rsp. You can click over rsp so IDA highlights everywhere it's used.

After a quick look, this seems interesting:

lea     rbx, [rsp+18h+var_18]

As noted before, it can be simplified to:

lea     rbx, [rsp]

Which is basically:

mov     rbx, rsp

Now we know that rbx is pointing to the top of the stack, and in the compare function, we see this:

add     rbx, 4
; ...
; ...
; ...
mov     [rbx-4], ecx

The first time that code is run, rbx points to the top of the stack; it's incremented once, but then the mov dereferences rbx - 4, which makes it be equivalent to:

mov     [rbx], ecx
add     rbx, 4

Which is essentially setting the element and skipping to the next one.

Now, since rbx points to the top of the stack, the first iteration will save the sum (i.e. ecx) to the top of the stack (which is the value you need to match), and increase the pointer to the next element (which you don't need here anymore).

Bingo, we've found out what's at rsp. It's just the value of ecx in the first iteration, which is basically the sum of the first group of characters plus one.

Next, notice it's using the ASCII codes of each digit instead of the digits themselves:

movsx eax, byte ptr [r9]

Since r9 points to a char array, the digit '1' is actually 49 (ASCII 49 = 1).

To understand it better:

char serial[] = "1111-2222-3333-4444";
int current_char = serial[0]; // puts the integer value 49, not 1
  • Awesome answer man, thanks a lot. Your number 1111-1111-1111-0000 does pass the check we've been talking about. I'm working on the next part of the puzzle. It's basically (add all 16 digits + 10) / 4. Compare this number to the sum of the first 4 PLUs 1, the second 4 PLUS 2, third 4 PLUS 3, and last four PLUS 4. I'll be sitting here fiddling with the numbers all night.
    – Asa Hunt
    Aug 24, 2015 at 1:59
  • @AsaHunt you need a full blown keygen, or a single valid serial?
    – rev
    Aug 24, 2015 at 2:04
  • @AsaHunt I just made this serial from the top of my head, can you check if it passes all the checks? 1111-1110-1100-1000 -- if it does, I can help you make a keygen for it
    – rev
    Aug 24, 2015 at 3:10
  • I just figured out the remaining checks! This was just an exercise I had to do for a job interview. Basically, figure out the algorithm that checks for valid serial numbers. Here's a break down: - 19 digits in the form xxxx-yyyy-zzzz-eeee - All 16 digits summed up + 10 and divided by 4 must equal the first 4 digits summed plus 1, second 4 digits summed plus 2, third 4 digits summed plus 3, fourth 4 digits summed plus 4. - The digits in the corresponding indices for 1st and 4th set can't be equal, and the same for the 2nd and 4th set. Lists don't work for me.
    – Asa Hunt
    Aug 24, 2015 at 3:28
  • I don't understand your last sentence; still, could you post the assembly of the code that does these checks? I think that's the best way of understanding it
    – rev
    Aug 24, 2015 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.