Need help extracting data from .dat file

Question

https://dl.dropboxusercontent.com/u/247623/item.dat This is a file from a Java game, and I would like to know how to legible string data from it.

I would prefer if only hints were given to me.

I have looked into it with a Hex Editor and come to the conclusion that the file is either compressed or encrypted because of the lack of a pattern.

If it is encrypted, I have no idea how to proceed. If it is compressed, I have found the following but have no idea how to proceed.

Entropy = 7.999055 bits per byte.

Optimum compression would reduce the size of this 384802 byte file by 0 percent.

Chi square distribution for 384802 samples is 505.10, and randomly would exceed this value less than 0.01 percent of the times.

Arithmetic mean value of data bytes is 128.1314 (127.5 = random). Monte Carlo value for Pi is 3.120328068 (error 0.68 percent). Serial correlation coefficient is -0.031097 (totally uncorrelated = 0.0).

Assume I have minimal knowledge of what I am doing, so several hints may be required. More information can be given upon request.

Thanks in advance!

Answer 1

Piece of cake: this is ZIP-compatible compressed compiled Java. Any familiar sequences of bytes near the header?

FA CE AF 0E 10 00 10 00 78 9C AC DD ..

.. those first few bytes look like Java's magic ID CA FE BA BE, but Googling that sequence doesn't yield any results, and it might be a red herring anyway. Next up: 10 00 10 00, which can be about anything (not a file length, but perhaps some internal flag or ID).

Next: 78 9C. These are familiar, they may form a set of ZLIB compatible compression flags. A quick-and-dirty ZLIB unpacker gave me (1) a positive unpacking result (meaning this data is valid compressed data and could be uncompressed), and (2) an output file much larger than the input (1,388,603 bytes vs. the original 384,802).

Inspecting the decompressed data I found this at the start:

01 09 00 31 5B 4C 63 6F 6D 2E 74 68 72 65 65 72 69 6E 67
73 2E 70 72 6F 6A 65 63 74 78 2E 69 74 65 6D 2E 63 6F 6E

or, in ASCII,

...1[Lcom.threering
s.projectx.item.con

-- enough to conclude it has been decompressed successfully. From this point on, you should be able to use common Java-decrypting/decompiling techniques.

Answer 2

It can be difficult to differentiate well compressed data (e.g., LZMA) from well encrypted data (e.g., AES). I've done a bit of analysis in the past however, and found that a lot of compressed data can be distinguished from encrypted data based on entropy analysis, as you have obviously already done.

However, I would take a simpler approach first. Look for common "magic" strings and/or headers in the file, such as those for gzip, zlib, and lzma. It may be that the game developers are using a standard compression, they've just added a custom header for their application.