I'm using IDA 7 Freeware, not Pro. I've been using IDC to re-invent the wheel somewhat, but I'm learning a ton of stuff as I go along.
I have a 16-bit MSDOS exe which I'm reversing. I stripped out the header and reloc table in my first attempt, so that all seg refs resolved correctly.
Now I've found set_fixup() and friends, and also seen doRelocs() from the (old) SDK via Google. I'm a bit puzzled by it all.
set_fixup() as per IDA help has 5 params. However set_fixup() expects 6 params in my version of IDA Freeware.
am I supposed to use set_fixup() on an entry in the exe header reloc table, or on the ea of an instruction in the program? (eg mov ax, 0x2d0e - where 0x2d0e needs to be adjusted to be 0x2d0e + 0x160 - code starts at 0x1600+10).
I can deduce from experiment set_fixup() looks like: set_fixup(targetEa, fixupTypeFlag, ???, ???, ???, displacement). Seems that p3 might be an ea, and p3 should be >p4 (otherwise nothing happens). p5 appears to have no effect!
doRelocs() - in the (ver?) dos_ovr loader - changes instruction bytes referenced by the exe header reloc table, physically changing the bytes in the code. It then also calls set_fixups(). I must be understanding fixups wrongly, or IDA fixups function differently to DOS exe relocations. Surely if you change the bytes to be relative to the start of the file ('mz'), then you no longer need to apply fixups?? They are fixed up by changing them! After all that's what DOS does when it loads the code before discarding the header.
Please help me understand IDA fixups!
I'm trying to use this IDC code:
//seg001:10 to_ea == 0x1610
//0x01610: mov ax, 0x02d0e (next: mov ds,ax)
//0x2d0e + 0x160 = 0x2e6e
set_fixup(0x1611, FIXUP_SEG16, 0x2e6e0, 0x2e6e, 0, 0) //best guess at args
Thanks in advance!
e: Partial answer from Ilfak Guilfanov (via email)
~~~~ Jul 18 (1 day ago) to me
Thanks, we will fix this. Meanwhile pass 0 for fixupf:
void set_fixup(long ea, long type, long fixupf, long targetsel, long targetoff, long displ);
Perhaps I'll be able to figure the rest out, but still grateful for any further help!