Dealing With x86 Segmentation Overlaps

Question

I load my project into IDA 7 and select 8086 as this rom is a 512kb in the high address space. I apply an offset of 0x80000 and it loads properly.

I then go to 0xFFFF0 and find a far jump to 0xFDAF:0x0 which has far jumps to 0xC000:0x0 and in that segment there are tons of 0xC000:0xNNN (lots of offsets).

It was a nightmare creating and shrinking segments so I greped through the binary dump and pulled the 8 hex values after every far jump (0xEA). I then formatted and calculated the ohysical addresses.

You can see in the segment list that the majority of segments overlap (eg):

0xC000:0x0000 = 0x0C0000
0xC000:0x12DB = 0x0C12DB
0xC003:0xA908 = 0x0CA938
0xC003:0xC908 = 0x0CC938
0xC007:0x506A = 0x0C50DA
0xC013:0x0903 = 0x0C0A33

From a running code prospective overlapping segments are irrelevent but IDA can not have overlaps. How am I suppose to walk through the code?

Files:
ROM.BIN
Segment List

---

0x49cdb: ea08a903c0 = jmp 0xc003:a908

00049cb0: 8b36 4a90 8b16 2990 2bf2 5e73 0b8b deb3  .6J...).+.^s....
00049cc0: 208b f3b0 08a2 8b90 f7c6 2000 7503 e9ab   ......... .u...
00049cd0: fd8b c6e8 9200 240f 3c07 747e a08a 903c  ......$.<.t~...<
00049ce0: 0774 05b0 00a2 8a90 a08b 903c 0874 03e9  .t.........<.t..
00049cf0: 8afd b200 b115 e8d7 45b0 d7e8 820d 8b36  ........E......6

Answer 1

So, a complete answer would be worth a series of blog posts but I'll try to touch the high points:

1. While you can use different seg:base pairs to refer to the same location, in real code it rarely happens. Code segments use the same base for all their functions and do not intersect with neighbors. However, they sometimes do not start or end at the exactly 16-byt aligned boundary. An example from this ROM:

   • segment with the base E0470 starts at E047:0008 (linear 0xE0478) and ends at E047:FF84 (linear 0xF03F4).
   • the following segment uses base F03F and thus starts at F03F:0004 (also linear 0xF03F4)

2. inside a code segment, near jumps and calls normally stay inside the segment, and the functions should not extend beyond the segment's end. This is one of the ways you can determine the real boundaries of a segment. For example, we have for the base E047 we have in the binary:

   • far call to E047:0008, so probably that's the start of the segment.
   • far call to E047:FF53 which is probably the last function of the segment. By disassembling the function and following only the near jumps, we come to the retf at 0xF03F2, or E047:FF82, so looks like the segment ends around there, and we can add the extra zero byte after it that was likely added by the linker for padding, arriving at the final boundary of E047:FF84 (linear 0xF03F4).

More extended example: after making an initial guess for segment's C84B boundaries to be C84B:0004 to C84B:F112, I noticed these instructions in the list of problems:

SEG_C84B:96A2 call    0F2D5h
SEG_C84B:EF69 jmp     0F395h
SEG_C84B:F0C8 call    0F3DFh

Since these are near calls and jumps, they should belong to the same segment. Following them gives us the extended end of the segment to be 0xd7894 or C84B:F3E4.

3. as an extension of above, data references using cs should point to data inside the segment, and it should make sense in the context. This is most visible with switch statements which look similar to following:

2E FF A7 B6 59  jmp     cs:off_B000_59B6[bx]
2E FF A4 8E 76  jmp     cs:off_C000_768E[si]

A good example is this one at 0xf05f1:

2E FF A7 06 02                 jmp     cs:206h[bx]
68 01                          dw 168h
6E 01                          dw 16Eh
74 01                          dw 174h
80 01                          dw 180h
86 01                          dw 186h

If we assume that the words following the jump is the jump table, then the segment should start around 0206h bytes earlier, and indeed there is a function start at 0xf03f4, so if we make a segment starting there (with base 0xf03f), our switch is nicely recovered:

SEG_F03F:01FD D1 E0          shl     ax, 1
SEG_F03F:01FF 8B D8          mov     bx, ax
SEG_F03F:0201 2E FF A7 06 02 jmp     cs:off_F03F_206[bx]
SEG_F03F:0206 68 01          dw offset loc_F03F_168  ; DATA XREF: sub_F03F_100+101↑r
SEG_F03F:0208 6E 01          dw offset loc_F03F_16E
SEG_F03F:020A 74 01          dw offset loc_F03F_174
SEG_F03F:020C 80 01          dw offset loc_F03F_180
SEG_F03F:020E 86 01          dw offset loc_F03F_186
SEG_F03F:0210 95 01          dw offset loc_F03F_195

I don't have the full picture yet, but the following boundaries seem to be close:

B000: 0xB0000-0xC0000 (full 64K)
C000: 0xC0000-0xC84B4
C84B: 0xC84B4-0xD7A52 (there is some weird stuff near the end)
E047: 0xE0478-0xF03F4
F03F: 0xF03F4-0xFDAF0
FDAF: 0xFDAF0-0xFFFF0

I added some scripts I used to my gist.

Note: some of the addresses printed out by the scanner are false positives and do not indicate real segments. You should consider only those that have multiple matches.

Data in lower segments seems to be compressed.

Answer 2

igorsk gave a nice reply there

this post is just to illustrate how you can use some standalone disassembler framework to look for proper ljmps instead of your xxd | tr hacks

I am using capstone below

PS F:\zzzz> Get-Content .\rombi.py
from capstone import *

md = Cs(CS_ARCH_X86, CS_MODE_16)
fin = open("f:\\zzzz\\rom.bin" , "rb")
buf = fin.read()
fin.close()
fsiz =  len(buf)
offset = 0
while (offset < fsiz):
        offset  =  buf.find(b"\xea",offset)
        if(offset != -1):
                patt = buf[offset:offset+5]
                for i in md.disasm( patt ,(0x80000+offset)):
                        print("0x%x:\t%s\t%s" %(i.address, i.mnemonic, i.op_str))
                offset = offset + 1
        else:
                break

running and analyzing the results

PS F:\zzzz> $foo = python.exe .\rombi.py

PS F:\zzzz> $foo.Count
484

last three far jumps
PS F:\zzzz> $foo[($foo.Count - 4)..$foo.Count]
0xfd7ce:        ljmp    0x2d20:0x4f00
0xfd7e9:        ljmp    0x2d20:0x5800
0xfdaf8:        ljmp    0xc000:0
0xffff1:        ljmp    0xfdaf:0

first three far jumps
PS F:\zzzz> $foo[0..3]
0x80162:        ljmp    0x507a:0xf0b8
0x8019c:        ljmp    0x4b48:0x4a6b
0x801fa:        ljmp    0xdae2:0xaa33
0x803c0:        ljmp    0x9b74:0xaa4a
PS F:\zzzz>