Background information: I have reason to believe that an old game I've played is still using their XOR encryption for nearly all in/out-going packets. I also believe that the key used in the encryption scheme remains static when I analyze the packet data, at least the frequency of certain values indicate so.
I know for a fact that previous keys has been 9-byte ASCII keys of length
Problem: I am not able to decrypt the packages. I've tried just directly looping the key, XOR'ing the data that is contained in the TCP packages and also tried the following C# function for XOR cryptography (See appendix).
Assuming Vigenere cipher is to be used as basis the previous keys suggest that the length might be 9 characters long.
I do have complete control of what the plaintext (ASCII in this case) of decrypted packages are gonna contain since I can use the chat functionality in the game. Thus I have tried sending the same message and observed the data difference within Wireshark as seen here:
Plaintext as sent in game:
In the following snapshots are three packets of the same message. Notice how the length remains the same, further indicating a static key length:
Overview (3 dots indicating data traffic)
Data side-by-side (Non-header data highlighted)
And finally: Hex dumps of the same packets. (See appendix)
I'm rather new to cryptography but find the field very interesting and hope that a kind spirit is willing to provide some assistance. Anything is appreciated.
Appendix code http://pastebin.com/F0py05Lz
Hex dump (due to low rep)
0000 aa 00 41 0e ad ce ff ce f0 ef e9 a0 f1 f8 9b a2 0010 fe df f8 e5 9c fb ed b8 a0 f6 f0 ca f7 ae d6 f8 0020 b4 9c fc e5 eb f7 a4 fb c8 a2 aa d5 f4 e1 cd a8 0030 e8 ef a2 a0 f8 c4 f1 fd 87 ff e3 c8 c3 e2 99 42 0040 de 51 ef 2b 0000 aa 00 41 0e ae 9a a8 ca f2 eb bf a3 a1 aa cf f5 0010 fa dd fc b3 9f ab bf ec f7 f2 f2 ce a1 ad 86 aa 0020 e0 cb f8 e7 ef a1 a7 ab 9a f6 fd d1 f6 e5 9b ab 0030 b8 bd f6 f7 fc c6 f5 ab 84 af b1 9c 94 d9 fd 32 0040 91 07 59 ba 0000 aa 00 41 0e af 9b fe 9a f9 e8 e3 f1 f2 a9 ce a3 0010 aa d6 ff ef cd f8 bc ed a1 a2 f9 cd fd ff d5 a9 0020 e1 9d a8 ec ec fd f5 f8 99 f7 ab 81 fd e6 c7 f9 0030 eb be f7 a1 ac cd f6 f7 d6 fc b2 9d c2 3f 2d 7a 0040 bd f6 42 d6