XOR encryption - How to find the key?

Question

I'm trying to solve a crackme which basically uses a XOR encryption scheme in which the password is used as the key to an encrypted buffer which is subsequently executed. The following is the encrypted buffer:

0x4F, 0x12, 0x85, 0x7C, 0x96, 0x2E, 0x29, 0x54, 0x45, 0x41, 0x15, 0x10, 

0xF1, 0x12, 0x87, 0x1C,
0xF3, 0x27, 0x27, 0x46, 0x56, 0x2D, 0x54, 0x2D, 0x41, 0x5D, 0x23, 0x31, 0x49, 0x5C, 0x2F, 0x21,
0x46, 0xF6, 0x92, 0x2E, 0x6D, 0x54, 0xBA, 0x90, 0x25, 0x23, 0x36, 0x21, 0x5C, 0x47, 0x21, 0x46,
0x4F, 0x46, 0x16, 0x94, 0x98, 0x2D, 0x01, 0x4D, 0xDC, 0xE0, 0xE7, 0x5C, 0x76, 0xE7, 0xC6, 0x4E,
0x46, 0x46, 0x2D, 0x3B, 0x83, 0xC1, 0x4F, 0x23, 0x31, 0x21, 0x29, 0xE9, 0xA1, 0x45, 0x4F, 0x46,
0x46, 0x0D, 0x92, 0xC5, 0x45, 0x4D, 0x23, 0x31, 0x45, 0x9A, 0xAF, 0x24, 0x46, 0x4F, 0x46, 0x2F,
0xEB, 0xD4, 0x43, 0x41, 0x4D, 0x23, 0x55, 0xE7, 0xDC, 0x28, 0x21, 0x46, 0x4F, 0x66, 0x80, 0xAD,
0x5C, 0x45, 0x41, 0x4D, 0x4A, 0xF7, 0xA1, 0x55, 0x2F, 0x21, 0x46, 0x3B, 0x80, 0xC6, 0x27, 0x54,
0x45, 0x41, 0x6C, 0xE5, 0xB1, 0x2A, 0x5C, 0x2F, 0x21, 0x46, 0x89, 0xC6, 0x56, 0x2D, 0x54, 0x45,
0x09, 0x8B, 0xA3, 0x20, 0x21, 0x5C, 0x2F, 0x40, 0x80, 0xCF, 0x54, 0x46, 0x2D, 0x54, 0x31, 0x87,
0xCD, 0x30, 0x31, 0x21, 0x5C, 0x5C, 0xE7, 0xC6, 0x5B, 0x46, 0x46, 0x2D, 0x74, 0x83, 0xC1, 0x58,
0x23, 0x31, 0x21, 0x33, 0xE9, 0xA1, 0x50, 0x4F, 0x46, 0x46, 0x4B, 0x92, 0xC5, 0x56, 0x4D, 0x23,
0x31, 0x47, 0x9A, 0xAF, 0x39, 0x46, 0x4F, 0x46, 0x66, 0xEB, 0xD4, 0x5C, 0x41, 0x4D, 0x23, 0x45,
0xE7, 0xDC, 0x35, 0x21, 0x46, 0x4F, 0x29, 0x80, 0xAD, 0x4F, 0x45, 0x41, 0x4D, 0x03, 0xF7, 0xA1,
0x40, 0x2F, 0x21, 0x46, 0x36, 0x80, 0xC6, 0x30, 0x54, 0x45, 0x41, 0x22, 0xE5, 0xB1, 0x3F, 0x5C,
0x2F, 0x21, 0x33, 0x89, 0xC6, 0x59, 0x2D, 0x54, 0x45, 0x61, 0x8B, 0xA3, 0x11, 0x21, 0x5C, 0x2F,
0x4C, 0x80, 0xCF, 0x67, 0x46, 0x2D, 0x54, 0x24, 0x87, 0xCD, 0x01, 0x31, 0x21, 0x5C, 0x5B, 0xE7,
0xC6, 0x6C, 0x46, 0x46, 0x2D, 0x31, 0x83, 0xC1, 0x69, 0x23, 0x31, 0x21, 0x72, 0xE9, 0xA1, 0x63,
0x4F, 0x46, 0x46, 0x0D, 0x92, 0xC5, 0x67, 0x4D, 0x23, 0x31, 0x71, 0x9A, 0xAF, 0x06, 0x46, 0x4F,
0x46, 0x2A, 0xEB, 0xD4, 0x6D, 0x41, 0x4D, 0x23, 0x54, 0xE7, 0xDC, 0x06, 0x21, 0x46, 0x4F, 0x27,
0x80, 0xAD, 0x7E, 0x45, 0x41, 0x4D, 0x50, 0xF7, 0xA1, 0x77, 0x2F, 0x21, 0x46, 0x2A, 0x80, 0xC6,
0x01, 0x54, 0x45, 0x41, 0x6D, 0xE5, 0xB1, 0x0C, 0x5C, 0x2F, 0x21, 0x35, 0x89, 0xC6, 0x68, 0x2D,
0x54, 0x45, 0x24, 0x8B, 0xA3, 0x1E, 0x21, 0x5C, 0x2F, 0x4F, 0x80, 0xCF, 0x76, 0x46, 0x2D, 0x54,
0x21, 0x87, 0xCD, 0x12, 0x31, 0x21, 0x5C, 0x0F, 0xE7, 0xC6, 0x7D, 0x46, 0x46, 0x2D, 0x2D, 0x83,
0xC1, 0x7E, 0x23, 0x31, 0x21, 0x33, 0xE9, 0xA1, 0x72, 0x4F, 0x46, 0x46, 0x58, 0x92, 0xC5, 0x74,
0x4D, 0x23, 0x31, 0x53, 0x9A, 0xAF, 0x17, 0x46, 0x4F, 0x46, 0x66, 0xEB, 0xD4, 0x72, 0x41, 0x4D,
0x23, 0x42, 0xE7, 0xDC, 0x17, 0x21, 0x46, 0x4F, 0x29, 0x80, 0xAD, 0x6D, 0x45, 0x41, 0x4D, 0x4F,
0xF7, 0xA1, 0x66, 0x2F, 0x21, 0x46, 0x3A, 0x80, 0xC6, 0x16, 0x54, 0x45, 0x41, 0x39, 0xE5, 0xB1,
0x1D, 0x5C, 0x2F, 0x21, 0x2F, 0x89, 0xC6, 0x7B, 0x2D, 0x54, 0x45, 0x2E, 0x8B, 0xA3, 0x0F, 0x21,
0x5C, 0x2F, 0x4F, 0x80, 0xCF, 0x79, 0x46, 0x2D, 0x54, 0x65, 0x87, 0xCD, 0x63, 0x31, 0x21, 0x5C,
0x5B, 0xE7, 0xC6, 0x0E, 0x46, 0x46, 0x2D, 0x3B, 0x83, 0xC1, 0x0F, 0x23, 0x31, 0x21, 0x7C, 0xE9,
0xA1, 0x05, 0x4F, 0x46, 0x46, 0x49, 0x92, 0xC5, 0x05, 0x4D, 0x23, 0x31, 0x42, 0x9A, 0xAF, 0x64,
0x46, 0x4F, 0x46, 0x34, 0xEB, 0xD4, 0x03, 0x41, 0x4D, 0x23, 0x50, 0xE7, 0xDC, 0x68, 0x21, 0x46,
0x4F, 0x25, 0x80, 0xAD, 0x1C, 0x45, 0x41, 0x4D, 0x48, 0xF7, 0xA1, 0x15, 0x2F, 0x21, 0x46, 0x7D,
0x80, 0xC6, 0x67, 0x54, 0x45, 0x41, 0x7D, 0xE5, 0xB1, 0x6A, 0x5C, 0x2F, 0x21, 0x76, 0x89, 0xC6,
0x0A, 0x2D, 0x54, 0x45, 0x76, 0x8B, 0xA3, 0x7C, 0x21, 0x5C, 0x2F, 0x61, 0x80, 0xCF, 0x08, 0x46,
0x2D, 0x54, 0x22, 0x87, 0xCD, 0x6C, 0x31, 0x21, 0x5C, 0x42, 0xE7, 0xC6, 0x1F, 0x46, 0x46, 0x2D,
0x35, 0x83, 0xC1, 0x1C, 0x23, 0x31, 0x21, 0x35, 0xE9, 0xA1, 0x14, 0x4F, 0x46, 0x46, 0x41, 0x92,
0xC5, 0x12, 0x4D, 0x23, 0x31, 0x0F, 0x9A, 0xAF, 0x75, 0x46, 0x4F, 0x46, 0x25, 0xEB, 0xD4, 0x10,
0x41, 0x4D, 0x23, 0x5E, 0xE7, 0xDC, 0x79, 0x21, 0x46, 0x4F, 0x2B, 0x80, 0xAD, 0x03, 0x45, 0x41,
0x4D, 0x23, 0x59, 0x21, 0x5C, 0x2F, 0x21, 0x16, 0xCC, 0x86, 0x56, 0x7D, 0x3C, 0x45, 0x41, 0x4D,
0x23, 0x88, 0x05, 0x33, 0x6F, 0x21, 0xB9, 0x9E, 0x85, 0x8D, 0x40, 0x00, 0x30, 0x74, 0x40, 0x00,
0x50, 0x73, 0x40, 0x00, 0x5C, 0x0F, 0x41, 0x00, 0x98, 0x73, 0x40, 0x00, 0x98, 0x72, 0x40, 0x00,
0x80, 0x74, 0x40, 0x00, 0xC0, 0x74, 0x40, 0x00, 0x28, 0x75, 0x40, 0x00, 0x0C, 0xD3, 0x41, 0x00,
0x34, 0x00, 0x46, 0x00, 0x50, 0x74, 0x40, 0x00, 0x94, 0xD3, 0x41, 0x00, 0xAC, 0xD3, 0x41, 0x00,
0x64, 0xD2, 0x41, 0x00, 0x28, 0x00, 0x46, 0x00, 0xD0, 0x73, 0x40, 0x00, 0xE4, 0xD2, 0x41, 0x00,
0x10, 0xE0, 0x45, 0x00, 0x38, 0x75, 0x40, 0x00, 0x44, 0x00, 0x46, 0x00, 0x08, 0x00, 0x46, 0x00,
0x2C, 0xD2, 0x41, 0x00, 0x8C, 0xD3, 0x41, 0x00, 0x10, 0x75, 0x40, 0x00, 0x7C, 0xD3, 0x41, 0x00

Upon receiving user input, the program checks its length and if it's different than 15 it aborts execution. So I know for a fact that the key is 15 characters long, so far so good. After the decrypting the buffer the following sequence of comparations takes place:

mov     eax, [ebp+decrypted_buffer]
mov     [ebp+var_4], eax
mov     eax, [ebp+decrypted_buffer]
cmp     byte ptr [eax], 60h ; 
jnz     short invalid_key

mov     eax, [ebp+decrypted_buffer]
cmp     byte ptr [eax+4], 0D0h
jnz     short invalid_key
mov     eax, [ebp+decrypted_buffer]
cmp     byte ptr [eax+9], 0
jnz     short invalid_key

Since the 1st, 5th and 10th bytes of the decrypted buffer are expected to be 0x60, 0x0D0, 0x0 we can find the 1st, 5th and 9th bytes of the key, namely: 0x2F, 0x46, 0x41

So the right key should be something like: "/***F****A*******"

My question is: How do I find the rest of the key?

Answer 1

You have to gather more constraints

• The key has to be 15 bytes long
• 1st, 5th and 9th bytes of the decrypted buffer are expected to be 0x60, 0x0D0, 0x0
• The Block should be executable!

And you should start with the last one. I assume this is an x86 executable, so 0x60 would correspond to an PUSHA instruction, which does does seem reasonable.

You could run a bruteforce with different keys and check if the result resolves to valid assembler, for example utilizing a stream disassembler such as diStorm or even CPU enumation with tools such as Unicorn.

Since you know 3 Bytes for sure, you have 12 bytes of entropy, resulting in 256^12 possible combinations to check. This is still too much, but you got even more information!

Since the xor key is repeated all 15 bytes, you already know 1/5 of all decrypted data. Have a look at it and try to see if you can make up reasonable contexts for it, such as common instruction sequences (prologes, epiloges, dataflow based).

For example, when the 5th byte is supposed to be 0xd0, this would evalute to a kind of shift operation. it that reasonable? When yes, where does the data come from? Maybe the four bytes between could indicate that.

Another point is the comment by sudhackar - What does the code do which yields the execution to this block? Which registers are set? Does it push a return location to the stack or is there a hard coded return location? Reversing this information would yield some bytes of the key.

edit:

Just realized you have even more information. When you limit the xor key to the range of printable ASCII characters, they will have to be in a range ox 0x20 to 0x7e. Reducing the base problem to 94^12.

Answer 2

I used a less smart approach. I wrote the hex blob to a file, wrote a quick Python script to add a padding byte after every 15th byte so the whole file is nicely aligned when opening it in a hex editor with standard 16 byte width and just had a look at it.

With the padding done, all columns share the same key so I just looked at what letter appeared the most in each of the 15 columns, hoping/assuming the most frequent byte in each column (in the plaintext) would be a 0 byte and thus leak a key byte.

Every column seemed to have a printable character as the most used which fits the context of a crackme where you enter a printable key, so it appeared this worked.

To verify, I googled for the supposed key and some writeup from 2008 had the same. If that writeup didn't exist I'd have to decode the blob and see if the assembly makes sense of course.