Help needed: Reverse engineer a DOS executable for a Game MOD

Question

befor you start reading, take a cup of coffee. I made a lot of text to explain my problem detailed. Because i can not load more than two links into this posting, i make a answer to show two additional pictures.

I'm Robert, mechanical engineer, and a fan of nostalig thinks (like cars or games). And thats the first step to my problem.

I want to modify my very first game i ever played. It's called "Radix - Beyond the Void". See here for more informations: https://goo.gl/Gy2kTw

I started reverse engineering the game about a jear ago. Within the time (maybe 30minutes a day) i was able to decode the game file and to view all the files the game is made with. A short discription:

The Game has two essential files:

• Radix.exe (the Game executable)
• Radix.dat (the Gamefile)

Radix.exe is loading parts from Radix.dat (like Sounds, images, levels) into the memory and than put it on the screen to move into the menu or -more important- to kill some enemies.

Radix.dat is a compressed gamefile with all necessary things for the Game. I used a HEX editor and Excel to read the index and i was able to extract all files. The Filetypes are:

• Music Files [.wav]
• Sound Files [.wav]
• Level Files [.lgd - own Radix format]
• Text Files [.txt]
• Image Files [own Format, see below]
• A Palette file for the Images [own Radix format]
• Demo Files (if you wait a minute, the games plays a demo) [own Radix format]
• Imagecontainer for Walls, Floors and Ceilings
• Imagecontainer for Things (like enemies or power ups)

The Imagecontainers have their own index and it was not a problem to extract the single images.

It took me some hours of work, i got that the images are indexed pictures. As example a 64px x 64px picture has exactly 4096 bytes. Every byte is a pixel and the decimal number of the byte is a position at the palette file for the colours. After some more hours i wrote a programm to show me the pictures with the right colours out of the palette file:

{See link 1}

The pictures for the game menu are indexed pictures too, but they are 800byte bigger than the resolution is. As example the Mainmenu has 320px x 200px and the file has a size of 64000byte + 800byte. The 800byte are splittet into 768byte for colors (256 red, 256green, 256blue) and some informations about position and size.

{See link 2}

If you stress Google for some minutes you will find a leveleditor to make you own Levels. Within the package with the leveleditor is a programm wich is called "3drescmp.exe".

3DResCmp.exe is needed to generate a new .dat File and play your own Levels in Radix. After i spend some hours with the 3DResCmp.exe i was able to change things like menuimages or Sounds, because the programmers didnt explain all functions of 3DResCmp.exe.

See here a DosBox Video: https://www.youtube.com/watch?v=YHL7irP7r3g

Now my mainly Problem: I'm able to generate a new Radix.dat-File except the Walls- and Thingsimages. This is because i dont know the original format of the images. The images are indexed, i can load the palette but not even one picture. I tryed at least 30 picture formats, tryed Photoshop 2.5 and 3.0 but nothing works.

Now my question: Can somebody reverse engineer the 3DResCmp.exe and can tell me the what format i have to use? I assume that 3DResCmp.exe take a image and create a indexed picture with the loaded palette file.

Radix, the Leveleditor and 3Drescmp.exe is written in Watcom C++ in Version of 1993 to 1995. I tryed IDA and some other tools, but i'm not a C++ programmer and i don't know how to interpret the results.

If you have questions or a tipp feel free to answer. I dont want that you make all the work, but i'm despairing at the images and need help from a software (reverse)engineer.

Greetings, Robert

Answer 1

I was able to decode the file format of the radix.dat file, which is mainly a container. It has a very small header, then the binary files concatenated to each other, then metadata directory at the end of the file (the header contains the file offset of the start of this metadata).

Most of the files are just that, binary files.

However, the two entries ObjectBitmaps and WallBitmaps are special, they act like directories, and have a collection of files inside themselves. Again, they consist of a small header, concatenated binary files, and a metadata directory. The file offsets in this metadata directory are relative to the outside radix.dat file, not to the Bitmaps file, so you have to extract those files when you extract the outer file; you can't extract ObjectBitmaps first and hope to extract the individual bitmaps later.

The WallBitmaps seem to be just a collection of pixels, 1 byte per pixel. The metadata has width and height entries, and in every single case, the size of the embedded file (calculated as distance between the start of the file and the start of the next file) equals width*height. Also, the code from 3dResCmp.exe seems to read 800 bytes from the file first, do some palette stuff, read the rest of the file, convert the rest by replacing stuff with a lookup table, and write the rest of the file. I didn't analyze this code completely, but it seems that it strips the 800 byte header from the file, converts the pixels from the file palette to a generic palette, and saves the pixels only, omitting the header/palette. You might be able to just throw one of your other image files at 3dResCmp for this.

The ObjectBitmaps are more complicated. The metadata structure is the same, but the size of the binaries does not equal widthheight. Instead, the binaries seem to start with what looks like coordinate pairs, one per row, 4 bytes per pair; then some pixels, but fewer than (widthheight). For example, an Object Bitmap named WeakBio-MineDead1 has 12x16 pixels, and 212 bytes. The binary starts with 16 coordinate pairs:

  64    1541
  70    2050
  78    2817
  89    2817
 100    2562
 110    2307
 119    2562
 129    2817
 140    3072
 152    2817
 163    2052
 171    2307
 180    2051
 188    1795
 195    2050
 203    2304

which are followed by 148 palette entries -- not enough for 12*16 pixels! (I'm not sure these are actually coordinate pairs, as the first x number seems to always be 4 times the number of rows).

The ObjectBitmap data is divided into two parts. The first part has one entry per row, denoting the byte position within the pixel map, number of "empty" pixels in that row, counted from the left, and number of "used" pixels. The second part is the pixel map, and it has 1 byte per pixel, just like the wall bitmaps.

For example, an object could use a 10(width)x4(height) bitmap and look like this:

.....XXX..
..YYXXX...
..YYX.....
..Z.......

with X, Y and Z being different palette entries, and . being "unused", i.e. transparent.

The bitmap format for this object would be:

0000    0x10 5 3        ;pixels start at 0x10, 5 unused pixels, 3 used pixels
0004    0x13 2 5        ;pixels start at 0x13, 2 unused pixels, 5 used pixels
0008    0x18 2 3
000C    0x1b 2 1
0010    58 58 58
0013    59 59 58 58 58
0018    59 59 58
001b    5a

Note that the size of the bitmap can, in this case, not be determined from the directory entry alone, as that has only width and height, not the byte size, which is generally much lower than (width*height).

When 3dResCmp.exe creates those ObjectBitmaps and WallBitmaps, it does remove the first 800 bytes from the input files, which means that even if you extract the files from radix.dat, you will not be able to put them back into your own level unmodified.

Anyway, i created a small program that extracts most files from radix.dat, and emits some information about the individual contents of WallBitmaps and ObjectBitmaps. Also, it can display an ascii-art version of an Object Bitmap. Feel free to look into it at https://github.com/gbl/RadixDatTool.