I would like to convert an instruction like:
lea ecx, [esp+150h+var_13E+2]
to:
lea ecx, [esp+14h]
I know it's possible to do this with the key K. But I'm writing an IDAPython plugin that needs an automatic conversion of instructions like the one above.
Edit: Are there any idaapi functions to do this?
idaapi.op_seg(idc.get_screen_ea(), -1)
idaapi.op_stkvar(idc.get_screen_ea(), -1)
Disclaimer: I am the author of the ida-minsc plugin (which aims to make idapython not suck) at https://github.com/arizvisa/ida-minsc
Using minsc, toggling the representation of an operand between hexadecimal or a frame variable can be done with instruction.op_hexadecimal or instruction.op_stack. These functions act on a single operand and take an address and an operand number.
Python>db.disasm()
'401304: lea ecx, [esp+14h]'
Python>ins.op_stack(h(), 1)
<member '$ F4012D0.var_284' index=5 offset=-0x288 size=+0x4 typeinfo='LPDWORD'>
Python>db.disasm()
'401304: lea ecx, [esp+298h+var_284]'
Python>ins.op_hex(1)
0x14
Python>db.disasm()
'401304: lea ecx, [esp+14h]'
Selecting these instructions within a function can then be done with the function.register function. Despite the semantics of Intel's lea instruction having the effect of doing SIB arithmetic and storing the result to a register, this can be treated as a load instruction using the %esp register. This should be parallel to the o_mem and o_displ operand types documented in the SDK as Operand types.
Python>for ref in func.register('esp', load=True):
Python> print(db.disasm(ref))
Python>
401304: lea ecx, [esp+14h]
However, it's pretty likely that we'd also want instructions that write to the %esp register as well. To enumerate both loads and stores depending on the %esp register (and eventually change the operand), we can chain them together using itertools.chain...essentially ORing them.
Python>for ref in itertools.chain(func.register(ins.reg.esp, store=True), func.register(ins.reg.esp, load=True)):
Python> print(db.disasm(ref))
Python>
40177f: mov dword ptr [esp+8], offset ConnectionHandler(x)
401787: mov dword ptr [esp+4], 0
40178f: mov dword ptr [esp], 0
...
4017a9: mov [esp], eax
401304: lea ecx, [esp+14h]
If you're not in a function (which makes no sense for frame variables), you can also use the instruction.ops_register function to filter an instruction's operands. By combining this with the database.address function, you can do a similar selection on any number of instructions that interact with registers in a particular way.
Python>for ea in db.a(function.chunk()):
Python> for ref in ins.ops_register(ea, ins.reg.esp, write=True, store=False):
Python> print([ins.op_repr(ref), db.disasm(ref)])
Python>
['esp', '4012d4: sub esp, 294h']
['esp', '4012da: and esp, 0FFFFFFF0h']
['esp', '401466: sub esp, 8']
...
['esp', '401714: sub esp, 4']
['esp', '40173a: sub esp, 4']
['esp', '40174e: sub esp, 4']
['esp', '40179b: sub esp, 18h']
['esp', '4017b1: sub esp, 4']
These results can then be combined with the instruction.op_stack function. The following python changes everything to a stack variable, outputs the reference, the result, and a disassembly of the modified instruction (for the purpose of demonstrating all of the available input types and their usage).
Python>for ref in itertools.chain(func.register(ins.reg.esp, store=True), func.register(ins.reg.esp, load=True)):
Python> lvar = ins.op_stack(ref)
Python> print(ref, lvar)
Python> print(db.disasm(ref))
Python>
opref_t(address=0x40176d, opnum=0, access=access_t(&w)) <member '$ F4012D0.arg_C' index=24 offset=0x10 size=+0x4 typeinfo='DWORD'>
40176d: mov [esp-4+arg_C], 0
opref_t(address=0x40177b, opnum=0, access=access_t(&w)) <member '$ F4012D0.envp' index=23 offset=0xc size=+0x4 typeinfo='const char**'>
40177b: mov [esp-4+envp], eax
opref_t(address=0x40177f, opnum=0, access=access_t(&w)) <member '$ F4012D0.argv' index=22 offset=0x8 size=+0x4 typeinfo='const char**'>
40177f: mov [esp-4+argv], offset ConnectionHandler(x)
...
opref_t(address=0x401787, opnum=0, access=access_t(&w)) <member '$ F4012D0.argc' index=21 offset=0x4 size=+0x4 typeinfo='int'>
401787: mov [esp-4+argc], 0
opref_t(address=0x401304, opnum=1, access=access_t(&r)) <member '$ F4012D0.var_284' index=5 offset=-0x288 size=+0x4 typeinfo='LPDWORD'>
401304: lea ecx, [esp+298h+var_284]
Similarly, it can also be combined with the instruction.op_hex function, or really any of the instruction.op_* functions.
Python>for ref in itertools.chain(func.register(ins.reg.esp, store=True), func.register(ins.reg.esp, load=True)):
Python> discarded = ins.op_hex(ref)
Python> print(db.disasm(ref))
40176d: mov dword ptr [esp+10h], 0
40177b: mov [esp+0Ch], eax
40177f: mov dword ptr [esp+8], offset ConnectionHandler(x)
...
401787: mov dword ptr [esp+4], 0
40178f: mov dword ptr [esp], 0
4017a9: mov [esp], eax
401304: lea ecx, [esp+14h]
