IDA ignoring register changes in pseudocode

Question

The Watcom compiler uses a fairly unusual calling convention, and IDA seems to be discarding some of the changes as irrelevant to its built-in pseudo-code. In my experience, that usually means I'm doing something wrong, as opposed to IDA :)

For example, in the following function fragment, the changes to ebx and edx are ignored in the pseudo-code.

; void __usercall RunScrIncDec(GeneralObject *object@<eax>, int *bufPtr@<edx>, int value@<ebx>)
RunScrIncDec    proc near
                cmp     byte ptr [edx], 0Ah
                jnz     short loc_164288

                inc     ebx
                inc     edx
                retn
...

Note here that bufPtr and value are both incremented, and value is indeed passed by value, not by reference.

If I change the function's return type so that it's an int by replacing the void return type with an int and appending the value location @<ebx>, then IDA includes ebx one in the pseudo-code, but still ignores edx.

Is there any way to tell IDA to pay attention to these changes? That edx and ebx aren't merely spoiled by the function, and that they're notable changes that should be reversed to bufPtr++ and value++ rather than not showing any pseudo-code at all? Or is this just something that IDA isn't built to handle?

Answer 1

As you rightfully figured out, IDA only takes into account changes it understands are related to the rest of the code. It will consider those values only in the case they're indeed return values the calling function.

What you'll need to do, as you've figured out yourself, is to make IDA understand those are returned values.

Here's a trick to let you do that, by setting that function to return a structure of two DWORDs (or any other type defined in the structure).

First, create an IDA structure by going to the structures view (shift+F9) and then create a new structure (INS).

In that structure, define two DWORD integers (either by using D on the bottom of the structure or CTRL+E to increase the structure's size first).

Then go back to the function, and modify the function's prototype. replace the void return type with the name of your structure, and append the value location specifier after the function's name. To specify two registers, in our case EBX and EDX`, use colons in between.

The final result should look like this, assuming you named your struct s_ret:

s_ret __usercall RunScrIncDec@<eax:ecx>(GeneralObject *object@<eax>, int *bufPtr@<edx>, int value@<ebx>)
^^^^^                        ^^^^^^^^^^