What exactly is this piece of assembly code doing?

Question

This assembly is for Intel x86-64 bit, seems to be too baffling to me.

• How come the jz instruction jump to a non-instruction (0x400AC9)?
• How come the call invokes a non-existing address?

For curious ones, the binary is here.

Answer 1

How come the jz instruction jump to a non-instruction (0x400AC9)?

It does not. There is no such thing as "an instruction". Jumps do not jump to instructions, they jump to addresses.

How come the call invokes a non-existing address?

It does not. If you check what the code actually does, you will find this call will never be executed.

---

Here is the relevant part of your code, prefixed with their original hex bytes.

400AC7:  66 b8 eb 05             mov    ax,0x5eb
400ACB:  31 c0                   xor    eax,eax
400ACD:  74 fa                   jz     0x400AC9
400ACF:  e8 0f b6 45 b0          call   0xffffffffb045b61c
400AD4:  3c 38                   cmp    al,0x38
400AD6:  0f                      .byte 0xf
400AD7:  85                      .byte 0x85
400AD8:  bc                      .byte 0xbc

If you assume the instruction at 400AC7 executes, it loads ax with some value but it gets immediately discarded by the next instruction, which clears it to 0. Because of that, the jump will always be taken!

The jump goes to 400AC9, and if we disassemble starting at that, we get some other code:

400AC9:  eb 05                   jmp    0x400AD0
400ACB:  31 c0                   xor    eax,eax
400ACD:  74 fa                   je     0x0
400ACF:  e8 0f b6 45 b0          call   0xffffffffb045b61a
400AD4:  3c 38                   cmp    al,0x38
400AD6:  0f                      .byte 0xf

and it looks like you did not get any further, because there are still weird calls and undefined codes. However, look at the first instruction! It jumps again into the middle of something what appears to be an instruction at this point – but if you start disassembling there, you will see that everything from that address on is perfectly ordinary code.*

^{* Since this is a "crackme" and you already failed the first of its tests, I will not spoil the rest of it.}

Answer 2

IDA disassembled the instructions incorrectly. You need to manually change the instruction after jz to data and then change the jz destination to code.

Answer 3

You should patch the ptrace the program uses to check if it's being traced by a debugger and then use gdb or another debugger to see what's going on @runtime, if you insist on statically analyzing it you'll have to manually fix up the jump as yujyet said.