(bad) opcodes of objdump

Question

I'm trying to write my own disassembler for PE,PE+ and ELF executables but I'm stuck with a big problem on PE and PE+ executables.

I'm checking my work by comparing my output with objdump, and I found some (bad) opcodes appear in the disassembled program. I immediately checked the instruction manual to control these values; they are shown as invalid in instruction manual. Some examples:

Example from PE files:

40dad1:       d6                      (bad)

Some other:

402f1c:       ff                      (bad)
402f1d:       ff                      (bad)
402f1e:       ff                      (bad)
402f1f:       ff 01                   incl   (%ecx) #at last a valid instruction

These are valid when I check the manual, but I cannot understand this (it's a PE+ file, architecture is AMD64):

f0 db a5 4e 9c 95 68    lock (bad) [rbp+0x68959c4e]

f0 is lock prefix
db means its a x87 instruction
a5 is ModRM byte(10 100 101) and by looking mod and reg fields we can say it's an invalid instruction
4e 9c 95 68 is used as 4byte disp but why ?

Do we assume that it's an invalid indirect x87 opcode and we continue to read as it's a valid opcode? I suppose objdump chooses this path.

And what are these (bad) instructions for? It's clear that they are not for aligning; or am I doing something wrong?

Btw, I'm trying to disassemble my old projects and FireFox to check if my program works. I'm using objdump -z -d -M intel XXYYZZ.exe to disassemble.

Answer 1

You are correct, and objdump is actually wrong to disassemble such instruction, only to mark it as (bad) in a later stage.

Here is how it works:

db a5 4e 9c 95 68    lock (bad) [rbp+0x68959c4e]

db is one of the 87 FPU extensions, and a5 indeed is the Mod/rm byte. Now, for the FPU extensions, the rm part is "as usual" for all other mod/rm instructions, but the mod part indicates which instruction to use, from this small table:

DB /0     FILD mem4i
DB /2     FIST mem4i
DB /3     FISTP mem4i
DB /5     FLD mem10r
DB /7     FSTP mem10r

where the /x number indicates what got encoded in the mod part of the mod/rm byte (for completeness: db eX also forms some valid opcodes).

So of all available codes, those with the mod/rm patterns of ..001..., ..100.. and ..110... form 'bad' codes - but you only know this after parsing the mod/rm byte and checking this specific table for the opcode DB.

Now apparently objdump parses the entire instruction - including the 4-byte immediate value - before checking if the base instruction is valid to start with. I suppose it's just a table that says

"fild", "(bad)", "fist", "fistp", "(bad)", "fld", "(bad)", "fstp"

and the (bad) entries get used as if they are actually valid.

One could argue that it's immaterial, because both ways lead to the conclusion the opcode is 'bad', but with objdump's method, you are not only discarding the first byte as 'bad' but an entire 6 bytes. It is at least theoretically possible the first byte (which causes the entire next sequence to be invalid) is data but followed immediately by correct code, which would then start with the sequence a5 4e 9c 95 68 - which is skipped entirely by objdump.

.. what are these (bad) instructions for?

I suppose, looking at the other instructions that you show, that you are disassembling a wrong part of the executable and this is not supposed to be code at all but data instead.

You should check the PE/PE++ headers to find the sections, and then only attempt disassembly on a section marked as "code" and/or "executable" in its Characteristics field. Even then, it's possible to start at a 'wrong' position (e.g., in the middle of a longer instruction) or inside data (which may also reside inside a .code section).