I got data which is compressed but I fail to find the compression algorithm. The data is part of a larger file from which I know the layout, So I managed to find out few things. What I know:
- I don't have the binary executable that load the data, I only have the updated version which no longer support the old copression algo. I tortured it in many way and it just doesn't contain the corresponding code
- it is compressed (100% sure of it)
- it can be home made as it was replaced later (see below)
- no magic numbers so far
- it is not plain:
- deflate (wrong header)
- lzma (wrong header)
- gzip (wrong header)
- Quantum (wrong header)
- Microsoft CAB (wrong header)
- Bzip2 (wrong header)
- Zip (wrong header)
- the uncompressed size is given in the file containing the data, this file layout is fully reversed and does not contain any clue
- it might be encrypted but is unlikely because of speed requirements
- if it is encrypted, it gives the same output given the same data input at the beginning of the sequence (by guess on some uncompressed data located nearby)
- it is from 2001 and has been replaced by deflate since
- Some of those data only output ASCII and nothing else (I know it from the layout of the container file) and have a compression ratio of about 0.30 (compressedSize/uncompressedSize) everytime
- I don't have any before/after data sadly
EDIT: There are the 32 first bytes in hex: b9daed36cb64bedb61b9dd2cb72afd8ee565b0dd2ea00f0afda2c36eb25b0016
I made histograms of several of those data and they all match a specific pattern. Something is going on with the powers of 2 obviously but I fail to see what.
Anyone has a idea what it could be? What can I do to gather further information? Does it look Lempel-Ziv based? If yes how could I reverse it?