Audio file compression identification

Question

I've been working on reversing file formats from the game of star wars episode 1: the phantom menace.

I have managed to extract the audio files from the .lab file in which they were contained. I cannot however play the audio files.

The files are provided at the end of the question.

I managed to find some of the decompressed files in the game's cache (which do play) and so I have a selection of audio files in both compressed and uncompressed form.

The compression ratio is about 66%; i.e. the compressed file is 1/3 of the size of the uncompressed size.

This is an example of the uncompressed and compressed form of the data. (From BDm8906.wav and BDM8906.wv)

The first 32 bytes of the uncompressed file (Magic is riff) BDm8906.wav

52 49 46 46 DA 77 00 00 57 41 56 45 66 6D 74 20
10 00 00 00 01 00 01 00 22 56 00 00 44 AC 00 00

The first 32 bytes of the compressed file (Magic is INDYWV) BDM8906.wv

49 4E 44 59 57 56 22 56 00 00 10 00 00 00 01 00 
00 00 F1 2B 00 00 00 00 00 00 B6 77 00 00 00 00

From looking a few of the compressed files I have found that:

The first 16 bytes are always the same. The next 4 bytes are the size of the compressed file. The next 6 bytes are always zero.

Here's a more detailed look at the hexdump of the compressed file:

https://i.sstatic.net/FDpGy.jpg

(SE will not let me upload it).

From byte 25 (start counting at 0) to the end of the file is always different. The number of bytes from here to the end of the file is the size of the compressed file indicated earlier.

Attempts to decompress:

1) Thorough googleing. One other guy on the lucas forums and on the xentax forums attempts to decompress it without success. It is not wavepack.

2) Trying common compression algorithms on the uncompressed files. Compress, lzma, gzip do not even come close to the compression ratio reported.

3) Binwalk on the binary:

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             Microsoft executable, portable (PE)
715764        0xAEBF4         YAFFS filesystem
721564        0xB029C         YAFFS filesystem
746196        0xB62D4         LZMA compressed data, properties: 0x64, dictionary size: 33554432 bytes, uncompressed size: 16777216 bytes

When I run binwalk on windows I get that it includes DMC (https://en.wikipedia.org/wiki/Dynamic_Markov_compression)

4) Dissembling the Binary and trying to find the location of the decompression code - this is quite difficult

A sample (two files, one encrypted one unencrypted) can be found here:

https://www.dropbox.com/sh/jvljza0v76mvl56/AAC41MenYP57dVHgAXY-DkSsa?dl=0

How can I figure out what it is using?

Thanks

Edit According to reaper, decompressed file:

22.05 sample rate 16 bits per sample 15323 samples 694 ms

whatever the game does has decoded it into 15323 16 bit integers

Answer 1

I was looking at the same problem and couldn't find a solution anywhere, so I implemented it myself!

smlu (Crt Vavros) had already started working on the format for his Indiana Jones and the Infernal Machine Mod Tools but sadly the format was slightly different: Indiana Jones files use WVSM decompression but on Phantom Menace the dialog files are compressed with a variant of ADPCM algorithm. My approach was to use Ghidra to disassemble and locate the algorithm, and also x64dbg for debugging.

I published everything on my github: IndyWV to WAV converter

I know it's a very old thread, but I hope this can help anyone who would like to extract the dialog files from Star Wars Episode 1: The Phantom Menace! And if anyone is aware of other games that also use that format, I'd be interested to know!