IDA SDK - can't understand the output of displacement query functions?

Question

OK so I want to parse x86 instructions with IDA SDK. However I can't understand the output, I have this code (that is executed after decode_insn at each operand that is present (type not 0)):

//op is an operand op_t of some insn_t
//ea1 is current address
                if(has_displ(op)) {
                    
                    msg("%x\n", ea1);

                    msg("%d\n", x86_base_reg(instr, op));

                    msg("%d\n", x86_index_reg(instr, op));

                    msg("%d\n", x86_scale(op));

                    msg("%d\n", op.addr);
                }

And this dissasembly:

.text:0040AE64                 lea     edx, ds:0[eax*4]
.text:0040AE6B                 lea     edi, [eax+eax*2]

And the output I get for each of the above instruction is:

40AE64
-1
0
2
0

For the second one it doesn't even get into the if. My question is why? And also why 2 in the output is not 4?

Additionally I would greatly appreciate if you could point me to some open source project that utilizes the API for fully parsing an x86 instruction, for reference.

Answer 1

Instruction encoding and decoding is one of the least user-friendly parts of the SDK. Since they have to support so many processor architectures with the same set of data types, they basically have to get creative with how they store information about the operands. It does not always make logical sense, but it really doesn't have to -- there are few requirements that really need to be satisfied beyond that the data can be stored and retrieved, and that the processor module knows what to do with it.

That said, you can gain familiarity with how things work on a given architecture by writing a plugin/IDAPython script to dump the operand information. I wrote such a script, which you can find linked in the second-to-last paragraph here.

To your specific questions:

1. I'm guessing that the second instruction is using an o_phrase operand, not o_displ.
2. The number 2 is representing the amount to shift left by, not to multiply by. This is consistent with how x86 encodes the *4 part of the memory expression.