Dear ReverseEngineering@SE,

Background information: I have reason to believe that an old game I've played is still using their XOR encryption for nearly all in/out-going packets. I also believe that the key used in the encryption scheme remains static when I analyze the packet data, at least the frequency of certain values indicate so.

I know for a fact that previous keys has been 9-byte ASCII keys of length

Problem: I am not able to decrypt the packages. I've tried just directly looping the key, XOR'ing the data that is contained in the TCP packages and also tried the following C# function for XOR cryptography (See appendix).

Assuming Vigenere cipher is to be used as basis the previous keys suggest that the length might be 9 characters long.

I do have complete control of what the plaintext (ASCII in this case) of decrypted packages are gonna contain since I can use the chat functionality in the game. Thus I have tried sending the same message and observed the data difference within Wireshark as seen here:

Plaintext as sent in game:


In the following snapshots are three packets of the same message. Notice how the length remains the same, further indicating a static key length:

Overview (3 dots indicating data traffic)

Data side-by-side (Non-header data highlighted)

And finally: Hex dumps of the same packets. (See appendix)

I'm rather new to cryptography but find the field very interesting and hope that a kind spirit is willing to provide some assistance. Anything is appreciated.

Thank you.

Appendix code http://pastebin.com/F0py05Lz

Hex dump (due to low rep)

0000   aa 00 41 0e ad ce ff ce f0 ef e9 a0 f1 f8 9b a2
0010   fe df f8 e5 9c fb ed b8 a0 f6 f0 ca f7 ae d6 f8
0020   b4 9c fc e5 eb f7 a4 fb c8 a2 aa d5 f4 e1 cd a8
0030   e8 ef a2 a0 f8 c4 f1 fd 87 ff e3 c8 c3 e2 99 42
0040   de 51 ef 2b

0000   aa 00 41 0e ae 9a a8 ca f2 eb bf a3 a1 aa cf f5
0010   fa dd fc b3 9f ab bf ec f7 f2 f2 ce a1 ad 86 aa
0020   e0 cb f8 e7 ef a1 a7 ab 9a f6 fd d1 f6 e5 9b ab
0030   b8 bd f6 f7 fc c6 f5 ab 84 af b1 9c 94 d9 fd 32
0040   91 07 59 ba

0000   aa 00 41 0e af 9b fe 9a f9 e8 e3 f1 f2 a9 ce a3
0010   aa d6 ff ef cd f8 bc ed a1 a2 f9 cd fd ff d5 a9
0020   e1 9d a8 ec ec fd f5 f8 99 f7 ab 81 fd e6 c7 f9
0030   eb be f7 a1 ac cd f6 f7 d6 fc b2 9d c2 3f 2d 7a
0040   bd f6 42 d6

1 Answer 1


The keys for your three messages appear to be cf7810d22, 42096edac and 4ea34873a respectively. (Note: those are 9-byte ASCII text strings, not hex numbers, even though clearly all the characters appear to be hex digits!)

OK, so how did I figure that out?

First, before even looking at the code you posted, I just took the messages from your hex dump, converted them back into binary (using a quick Perl script), and XORed them together. The resulting XORed messages look like this:

$ perl -0777 -E '$a = <>; $b = <>; print $a ^ $b' packet1.dat packet2.dat | xxd
0000000: 0000 0000 0354 5704 0204 5603 5052 5457  .....TW...V.PRTW
0000010: 0402 0456 0350 5254 5704 0204 5603 5052  ...V.PRTW...V.PR
0000020: 5457 0402 0456 0350 5254 5704 0204 5603  TW...V.PRTW...V.
0000030: 5052 5457 0402 0456 0350 5254 573b 6470  PRTW...V.PRTW;dp
0000040: 4f56 b691                                OV..

XORing the ciphertexts together like this cancels out the plaintext (assuming that it's the same in both messages), leaving just the XOR of the keys. We can see that there's a clear repeating 9-byte pattern in the XORed data, strongly suggesting that the messages have indeed been encrypted with a repeating 9-byte key.

Now, if that was all, we could find the key simply by taking the known plaintext string TestingTheKeyTestingTheKeyTestingTheKeyTestingTheKey, XORing it with the ciphertext at different positions (since I didn't know exactly where the known plaintext would occur in the encrypted message), and looking for a result that looks like a plausible key. In crypto jargon, this method is known as crib dragging, a term that dates back at least to Bletchley Park during WWII.

Alas, when I tried that, it didn't yield anything that looked anything like a repeating 9-byte text string, as I expected the key to be. In fact, there were hardly any printable ASCII characters in the output at all. Looking more closely at the ciphertext, I noticed that most of its bytes had the high bit set, something that can't happen when you XOR two ASCII characters together. So clearly there had to be something else going on, too.

At that point, I took a closer look at your (presumably previously reverse-engineered) decryption code, and realized that it actually XORed the message with three different byte streams:

  1. the repeating 9-byte key (applied starting from the sixth byte),
  2. the fifth byte of the ciphertext (Incrementor, presumably some kind of a message counter; this is what caused the high bits of each byte to be set, since in your messages it had values from 0xAD to 0xAF), applied to every following byte, and
  3. a byte value (KeyVal) that starts at zero and is incremented by one every time the key repeats, i.e. every ninth byte after the sixth.

(The code also skips step 3 if KeyVal equals Incrementor, presumably to prevent the last two steps from canceling each other out. That little quirk actually has little if any cryptographic significance, and in any case, it will never happen for these short messages with high Incrementor values.)

Now, conveniently, the XOR operation, like addition, is commutative and associative — that is, if you XOR two or more bytes together, it doesn't matter which order you do it in: (A XOR B) XOR C == A XOR (B XOR C) == A XOR (C XOR B) == (A XOR C) XOR B == .... Thus, since I knew the Incrementor value for each message (since it's given in the encrypted message) and had a pretty solid guess for the repeating key length (which let me calculate KeyVal for each byte), I could just XOR each encrypted byte with those two values, leaving me with just the plaintext XORed with the repeating key.

After that, it was easy enough to discover by crib-dragging that the known plaintext string actually starts two bytes after the beginning of the encrypted portion (and so seven bytes after the beginning of the whole message), and obtain the 9-byte key for each message.

(What I can't tell from just your three sample messages is how those 9-byte key strings actually are generated, or how the game knows which key string to XOR a given message with. For that, you may need to analyze more message packets and see if you can find any kind of pattern to the keys.)

  • Before I go in-depth with a couple of questions I'd like to let you know how I truly appreciate your elaborate response. The effort you've put into explaining the process is worth so much more than the results by themselves. If you were to give me one advice on further development of this work, how would you go about it? Look at the ASM in a debugger? More packets? Also: On my Linux box I run perl (5.20.2), but when I copy your bash line, perl produces the following pastebin.com/rJP1TKTU My dat files are just copies of each individual dump in hexadecimal. Do you notice anything off?
    – Syncretic
    Feb 8, 2016 at 16:42
  • 1
    I'd start by grabbing more packets of known plaintext first, and trying to see if I could spot a pattern to the keys. If not, debugging and disassembly would be my second choice. Also, my .dat files were in raw binary, not in hex. (I think I used something like perl -nE 's/^\S+//; s/\s+//g; print pack "H*", $_' < hexpacket.txt > rawpacket.dat to convert your hex dumps to binary.) Feb 8, 2016 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.