Wrong relocations on programs requiring DOS/4G (or DOS/32) 32-bit extender

Question

In the last week I've attempted maybe 3 times to debug a game in DOSBox (Fallout 1 DOS version), and the game uses DOS/4G to extend itself to 32-bit protected mode. The program has a DOS stub which then calls DOS/4G with the game as parameter, which then goes directly for the Linear Executable (LE) header and starts the program there.

I can debug a program with no problems if DOS/4G is not used. Though, from the moment DOS/4G initiates the game, the addresses go nuts and I don't understand why it happens.

From what is on the header, the code segment should start at 0x44000 and have size DB000 (so going up to 0x11EFFF). So far so good, except that on DOSBox Debugger, addresses go to 7 digits, and also happens on Oracle VM VirtualBox with actual MS-DOS 6.22 with OpenWatcom Debugger.

So, does anyone know if DOS/4G or DOS/32 change the relocation addresses? Wouldn't that break the program anyway because of references from different sections? (I'm kind of a beginner on RE)

I'd at least like to know where I am when debugging. If I'm on “random” addresses, I can't compare with IDA and see where I am and go to some address because I've got exactly no idea where the code I see on the debugger is. On OpenWatcom Debugger I don't see a hex dump, but on DOSBox Debugger I do. I've tried to search for the hex values and nothing came up on HxD --> ????? This is confusing, can't be that hard to debug a DOS/4G program.

Any help on understanding what's happening here is appreciated!

Answer 1

It seems that after a week and hints from other places where I asked this (this may be the 5th place?), and after an experiment with a small program for DOS I compiled with OpenWatcom bound to the DOS/4G extender, I remembered there are breakpoint instructions. So I used one of them, 0xCC, and DOSBox Debugger stopped in the beginning of main(). Now I can know where on IDA is something I see on the debugger.

In this case the debugger wrote this message: Breakpoint trap (INT 3) at <0830:001928A8>, unrelocated address <1:00062898>. On IDA, main() begins at 0x72898. So, from here, 0x1928A8 - 0x72898 = 0x120010 and there's the address where the code section begins on the debugger. Then it's just sum 0x120010 to the IDA addresses to know the debugger equivalent, and subtract 0x120010 to the debugger addresses to know the IDA equivalent. To know where other sections are, I just find some instruction that references them and do the same math. Like here on the debugger an address of a string on the data section is 0x214F6C. On IDA is says that should be 0xF9F4C, so the beginning of the data section is 0x214F6C - 0xF9F4C = 0x11B020.

---

Still though, if anyone knows, I'd like to know how I can know in a more decent way where the programs are located. On that experiment I made, the code section was on 0x14D000. The data section should be 0x10000 after the code one, but it was not, because it started on 0x163000, so it's not linear as I would expect (actually I would expect the addresses to be those on the file header - not sure why the program is re-relocated or something).

And going back to that data section string address, the instruction is mov eax, 0x214F6C on the debugger, while on IDA (the correct one, I'd say) is mov eax, 0xF9F4C --> DOS/4G modifies the program when it's loading it!? Because this is an hard-coded address, not offset!