3

I'm reverse-engineering a DOS binary that was compiled with Borland C++ 3.0, and I'm trying to create C code that will produce the same assembly as the original when compiled with the same compiler. For the majority of the code so far, this works out fairly well, but there is one place that's really puzzling. I just can't figure out what kind of source code would get the compiler to produce this particular asm sequence.

So the relevant part of the original assembly is:

; function's stack layout:
vram_ptr        = dword ptr -4
target_offset   = word ptr  6
size            = word ptr  8
src_data        = dword ptr  0Ah

; the funtion also uses registers SI and DI to hold two loop variables.

; ----------------------------------------------------
; The code in question. It's basically just
; vram_ptr[i] = *src_data, but done in a very
; convoluted manner
; ----------------------------------------------------
les     bx, [bp+vram_ptr] ; load vram_ptr into ES:BX
push    es

mov     es, word ptr [bp+src_data+2] ; set ES to FP_SEG(src_data)
push    bx
mov     bx, word ptr [bp+src_data] ; set BX to FP_OFF(src_data)
mov     al, es:[bx]     ; basically _AL = *src_data
pop     bx

pop     es
mov     es:[bx+si], al  ; Store it
; ----------------------------------------------------
; ----------------------------------------------------

; src_data++
inc     word ptr [bp+src_data]

When I do the most obvious thing and write dest[i] = *data; in C, the compiler generates something much more reasonable, but sadly different from the original:

les  bx,[vram_ptr]   ; same as original 
push es              ; same as original

les  si,[src_data]   ; different, original avoids
mov  al,[es:si]      ; use of SI

pop  es              ; same as original
mov  [es:bx+di],al   ; same as original
inc  word [src_data] ; same as original

And it also spills a loop variable to the stack that is held in SI in the original code, and uses DI instead of SI for the loop variable used in the expression. But no matter what I try, I can't get it to generate the original assembly.

It seems as though the compiler was somehow prevented from using SI in the data move expression, and was thus forced to generate the more complicated sequence.

What kind of source code and/or compiler flags might cause this?

For reference, here's the full C code I have so far (which is close to the original asm aside from the mismatches I mentioned):

void pascal CopyToVram(byte far* data, word size, word destOffset)
{
  register word i;
  register word mask;

  byte far* dest = MK_FP(0xa000, destOffset);

  // These macros just generate a bunch of port IO instructions
  EGA_SET_DEFAULT_MODE();
  EGA_SET_DEFAULT_BITMASK();

  for (i = 0; i < size; i++)
  {
    for (mask = 0x0100; mask < 0x1000; mask <<= 1)
    {
      disable();

      outport(0x03c4, mask | 0x02);

      // TODO: Codegen mismatch
      dest[i] = *data;
      data++;

      enable();
    }
  }
}
8
  • 1
    May be the code wasn't compiled at all and is inline assembler? Borland C++ 3.0 is from 1991 and at that time hand written assembler code wasn't still popular with lots of people.
    – Robert
    Jul 6, 2022 at 16:14
  • @Robert I was wondering this as well, but I doubt it. There is definitely handwritten asm in this executable, but it's usually easy to tell apart from the compiled code. The rest of this function's asm is too close to the compiler output I get when compiling it to be entirely handwritten, and I would be surprised if someone wrote inline asm just for one assignment. Jul 6, 2022 at 16:21
  • Isn't it possible to include blocks of assembler into a common C function using inline assembler (never used it, I I thought it would work this way)? Therefore not the whole function needs to be handwritten assembler.
    – Robert
    Jul 6, 2022 at 16:31
  • @Robert it's definitely possible, but I would find it strange that someone writes this complicated inline assembly just to do a single assignment, where the compiler already does a good (better?) job. Inline asm within functions is typically used for performance critical parts, or very low-level things. If someone went to the trouble of optimizing this function with inline asm, I would expect to see things like REP STOSB and the like. Jul 6, 2022 at 16:37
  • What if you use indexes for both source and destination pointers?
    – Igor Skochinsky
    Jul 6, 2022 at 21:16

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.