Difficulty decrypting LCG string

Question

I'm trying to analyze some malware. To get the hang of it I am following the decoding process on this website:

Here is a description of the string encoding/decoding process:

String encoding utilizes LCG fed by the new PRNG algorithm, while the generated keys are then subtracted from each ciphertext byte to generate the plaintext string. The first DWORD in the encoded string is used as the seed, while that same value is XOR’ed against the second DWORD to calculate the size of the encoded string. The encoded string begins at the position after the second DWORD. Most suspicious strings in the unpacked DLL are encoded using this method.

Here is the python PRNG method provided by the article that I am using:

def prng(seed): return ((seed * 0x41C64E6D) + 0x3039 ) & 0xFFFFFFFF

Then you are supposed to decrypt some encrypted traffic using a PHPSESSID cookie encoded with this scheme:

I am only trying to turn the encoded cookie into the decoded cookie. However, I cannot replicate the results on my own. My first problem is that I cannot seem to find the initial correct seed. I know it is supposed to be the first dword, but is it the dword of the cookie as it is (5C8EC19e) or, since this cookie is in ASCII, is the dword referring to the hex value of "5C8E"(35433845)? I have tried both ways, using the given algorithm, and cannot match the decoded string.

I'm also confused because it seems like I can get pretty close to the decoded value by not using dwords at all. If I xor the first (5C) and second(8E) bytes of the encrypted cookie, I get D2, which is the appropriate number in the decoded cookie. If I continue xor-ing 5C with the next two numbers, C1 and 9E, I get 9D and C2, which are one off from the actual decoded values. Is this just a coincidence?

TL;DR: The PRNG algorithm is provided as the appropriate decryption function by the article I am following. The body of the HTTP message is encrypted using an RC4 key. The RC4 key is encoded within the PHPSESSID cookie. I believe, based on what I read in the article, that the RC4 key is decrypted from the PHPSESSID cookie using this PRNG function. It may be that the PRNG function is not the correct function for this - I can't get it to work. I just want to know how the encoded PHPSESSID is becoming the decoded PHPSESSID

Answer 1

I found this paper that shed some light onto what's happening:

https://www.blueliv.com/downloads/technical-report-vawtrak-v2.pdf

So I took the PHPSESSIONID from your screenshot and wrote this script:

import binascii

data = "5C 8E C1 9E 61 66 6B 71 7F 80 8B 93 9E AA B7 C5"
data = bytearray(binascii.unhexlify(data.replace(" ","")))

k = data[0]

for i in range(1,len(data)):
    data[i] ^= k
    k += i

print(binascii.hexlify(data[1:]))

which prints

$ python test.py 
b'd29cc1030000000700020000000000'

and makes sense. The first 4 bytes then are the RC4 key.