Figuring out the CRC part of a packet

Question

Currently, I am trying to figure out the packet structure (for encryption) of an old MMO in order to create an emulator. Its official servers shut down 3 years ago and this version is over 11 years old.

After loading up the original server (it takes up ~3gb of RAM just to load and eats quite a lot of CPU, so that's why I want to emulate it) with IDA, I found that the polynomial is 0x8005 (32773) and that it was "Crc16". To further back that up, I found a series of bytes in memory when it's loaded:

00 00 [05 80] 0F 80 0A 00 1B 80 1E 00 14 00 11 80 33 80 36 00 3C 00 39 80 28 00

And, from IDA (pInitial is always 0 for some reason):

FnlApi::CCrc16::SetPolynomial(&this->__mCrc, 32773);
//////////////////////////////
v5 = FnlApi::CCrc16::CalcCrc(&this->__mCrc, pPack + 2, pLng - 2, 0);
//////////////////////////////
int result; // eax@1
const char *v5; // edx@1
__int16 v6; // bx@2
unsigned __int16 v7; // bx@2
bool v8; // zf@2

LOWORD(result) = pInitial;
v5 = pMsg;
do
{
    LOBYTE(v6) = 0;
    HIBYTE(v6) = result;
    v7 = this->__mTbl[(unsigned __int8)(*v5++ ^ BYTE1(result))] ^ v6;
    v8 = pLng-- == 1;
    result = v7;
}
while ( !v8 );
return result;

This is a packet from the server that isn't encrypted.

[1B 00] [00] [00 F2 2C 00 00 00] [31 32 37 2E 30 2E 30 2E 31 00 00 00 20 01 00 00 16 27]

As I understand, this is the packet "structure":

[Packet ID] [Key (00?)] [CRC, padding?] [Actual packet]

This is a packet from the server that is encrypted.

[2F 00] [74] [38 ED 2C 00 00 01] [93 85 AE 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A F3 3C EF EF F9 AE 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 9A 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]

Decrypted, this is (I think):

[2F 00] [74] [38 ED 2C 00 00 01] [75 69 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 70 61 73 73 77 64 00 00 00 00 00 00 00 00 00 00 00 00 00 CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC CC]

I think both of the packets's structure is:

[Packet ID] [Key] [CRC, padding?] [Actual packet]

Another packet that is encrypted with the same structure:

[18 00] [42] [56 D6 07 00 00 01] [2E 75 3D D2 75 03 CC 03 CD CC 03 8E 2D CC B1 6B 9F DB 6B D1 DB 6B 4F 4F D3 73 78 C6 AA 4B DF 63 1E 4A 18 C6 23 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]

Decrypted:

[18 00] [42] [56 D6 07 00 00 01] [63 74 65 73 74 00 01 00 08 01 00 04 02 01 03 C7 0F AC C7 C4 AC C7 A3 A3 E7 5C 7F 05 CA 6D A0 77 46 64 F8 05 9E 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F]

I apologize for this being a mess, but I've been at it for days with no actual progress.

Edit:

My current class (with decryption working) is here: http://pastebin.com/7e05AJLa

Answer 1

The CRC is, as suspected, a fairly standard CRC16 with the polynomial 0x8005 (reflected: 0xA001). The only online calculators that I could find for this are:

• BobTech's (clear the 'reverse data bytes' and 'reverse CRC' check boxes)
• GHS Infotronic's (the link has polynomial and message "123456789" already inserted)

The latter is particularly convenient for taking parts of the packet hex dumps, CRCing them, musing upon the results and then changing some bits before repeating the experiment...

Except for the polynomial - 0x8005 instead of 0x1021 - all the specifics are the same as for CRC-CCITT (XModem). If you plug in the 0x1021 poly then you can use a multitude of online checkers for verifying your own implementation, like Lammert Bies's, or indeed FoxPro's builtin CRC function (with 0x10000 as seed to force 0 as initial value, e.g. sys(2007, "123456789", 0x10000)).

Conversely, there are plenty of ready-made implementations where you only need to substitute 0x8005 for 0x1021. I've also found a table-based implementation in C that already uses 0x8005.

The code in the disassembly is essentially the same as that in chapter 10 of the CRC bible, a.k.a. Ross Williams' Painless Guide to CRC Error Detection Algorithms (HTML version here):

r=0; 
while (len--) 
    r = (r<<8) ^ t[(r >> 8) ^ *p++];

This is readily apparent if you compare this to my transcription of the disassembly (which I did in C#, because I'm learning it ATM):

static ushort crc16 (byte[] pMsg, ushort pInitial = 0)
{
    uint eax = pInitial;

    for (int i = 0; i < pMsg.Length; ++i)  // EDX == pMsg + i
    {
        uint cl = ((eax >> 8) & 0xFF) ^ pMsg[i];
        uint bx = (eax & 0xFF) << 8;

        eax = bx ^ lookup[cl];
    }

    return (ushort)eax;
}

I've shown an intermediate rendition that still has the registers as value names, which makes it easily to correlate to the disassembly.

Sidenote: the disassembly shows IDA's age-old bug whereby it locally renames registers to the names of the parameters they hold on entry to the function.

E.g., for reading the disassembly of this particular function you need to substitute 'ecx' for 'this' mentally because otherwise it just doesn't make sense:

    ...
    xor this, this
    mov cl, ah
    xor cl, bl

    xor ebx, ebx
    mov bh, al
    and this, 0FFh
    movzx   this, cx
    xor bx, [esi+this*2]
    ...

It's simply ludicrous. I guess a certain someone needs to read up on the concept of live ranges, and why compilers bother to add live range data to the bits of debug info where IDA gets the name/register association from.

For completeness' sake here are the other parts of the CRC rig, for those who want to use C# for spelunking:

static ushort[] lookup;

static void initialise_lookup (ushort polynomial = 0x8005)
{
    for (uint edx = 0; edx <= 0xFF; ++edx)
    {
        uint ax = edx << 8;

        for (uint esi = 0; esi < 8; ++esi)
            if ((ax & 0x8000) != 0)
                ax = (ax << 1) ^ polynomial;
            else
                ax <<= 1;

        lookup[edx] = (ushort)ax;
    }
}

static ushort crc16 (string s, ushort initial = 0)
{
    return crc16(System.Text.Encoding.ASCII.GetBytes(s), initial);
}

That's it for the raw basis of the packet CRC. Now it's time to hammer out the specifics and to see if someone deviously passes something other than 0 for the initial value, or whether the resulting CRC is stomped on somehow before being inserted into the packet... Another thing to watch out for in the original EXE is that the CRC class might well get instantiated with different polynomials in different parts of the program.

Eyeballing the packets in WPE_Plogtxt.txt shows that only the third and fourths bytes (offsets 2 and 3) have enough entropy to be a CRC, and that the initial word is the length of the packet:

[u16 length] [u16 crc?] [5 headerish bytes] [payload]

Examples:

09 00 : 01 B9 : DF 07 00 00 01
09 00 : E2 CD : 75 00 00 00 01

0B 00 : 12 AD : 10 00 00 00 01 : 95 03 
0B 00 : 14 05 : 10 00 00 00 01 : A9 9A
0B 00 : 11 2B : 10 00 00 00 01 : BA CD
0B 00 : 77 D0 : 46 00 00 00 01 : 4D 2A

0C 00 : 2D B1 : 42 00 00 00 01 : B7 D1 90
0D 00 : 04 E6 : 6D 01 00 00 01 : F1 52 ED DF
0E 00 : 50 9D : 2C 00 00 00 01 : F7 E9 94 FC 3A
0F 00 : 4C 88 : 21 01 00 00 01 : 30 EB 62 D0 C4 D0
10 00 : 81 28 : 1B 00 00 00 01 : 64 CD E6 F0 24 CD F0
11 00 : C5 FB : 31 00 00 00 01 : 75 0F 13 10 DE 0F 11 2F 
12 00 : 9E F7 : 18 00 00 00 01 : 19 D8 F5 D8 F2 01 95 D8 D8

But sometimes the stuff doesn't fit at all:

13 00 : 00 00 : 03 00 00 00 00 : 00 00 0B E1 F5 05 00  00 00 00

In order to make quick headway it would be helpful to run analyses on large numbers of samples, i.e. directly on nice fat packet capture files. Where's this "PAC" format from?

Here are all samples of length 9 that I could find, to demonstrate why more analysis is necessary. Several of the samples differ only in the supposed CRC. This means that if these frames do contain CRCs then these must refer to some bigger frame of reference.

09 00 : C5 28 : 05 00 00 00 01
09 00 : CD 65 : 05 00 00 00 01
09 00 : EA A6 : 1E 00 00 00 00
09 00 : 58 29 : 2D 00 00 00 00
09 00 : CC 96 : 2D 00 00 00 00
09 00 : D0 B0 : 2D 00 00 00 00
09 00 : F8 8A : 2D 00 00 00 00
09 00 : 57 3C : 4F 00 00 00 01
09 00 : 41 47 : 5A 00 00 00 00
09 00 : 8E A2 : 5A 00 00 00 00
09 00 : DF B3 : 5A 00 00 00 00
09 00 : E2 CD : 75 00 00 00 01
09 00 : 01 B9 : DF 07 00 00 01
09 00 : 02 C9 : DF 07 00 00 01
09 00 : 0F 39 : DF 07 00 00 01
09 00 : 1A F2 : DF 07 00 00 01
09 00 : 36 7E : DF 07 00 00 01
09 00 : CD 65 : DF 07 00 00 01
09 00 : D0 55 : DF 07 00 00 01
09 00 : D2 D9 : DF 07 00 00 01

This complicates the picture and makes it more difficult to work out the specifics of the CRCs via packet analysis. It might be faster to attack the problem with IDA, by analysing references to the CRC function.