IDA - Some issues regarding function's/data/code recognition

Question

I have to write an IDAPython script and I came across many false positives in IDA's function/code/data recognition.

Case 1 - Invalid function's end (rpcrt4.dll):

In some DLLs (especially in rpcrt4.dll) IDA recognize indirect/dereference calls as function's end and then misidentifies the next instructions (IDA's no-return kernel analysis flag is of course disabled so it is not the problem).

Example (the unknown data is a valid jmp instruction):

Case 2 - Data in .text section is identified as code (ExplorerFrame.dll):

Most of Microsoft's DLLs are compiled with data at the beginning of the .text section and IDA successfully identifies it as data. However, sometimes IDA fail to treat these heads as data and treats them like code (my guess is that it has something to do with the misidentified ret instruction).

Example:

Case 3 - code inside a function is identified as data (mimeTools.dll):

This case is a little bit weird because I don't see any reason to make this mistake. As you can clearly see IDA identifies code as data inside the function (the misidentified code is a lea instruction).

Example:

Case 4 - mixed misidentified code (igc32.dll):

No need to elaborate, in the following example it should be all code but it gets identified as data:

And another example:

So obviously my question is: did anyone experience one of these problem's before and know a possible fix to some of these problems? I need a solution that don't involve human interaction and can be applied using an IDAPython script.

Thanks in advance.

Answer 1

Disassembling an executable is hard. It is especially hard for machines to detect edge cases and similar nuances about small portions of code.

This is something IDA (and any other complete disassembler) puts a lot of effort into, but based on design decisions and heuristics used in different disassemblers, results vary. IDA has seen a consistent improvement in addressing such as similar issues throughout version updates.

It is however unrealistic to find and properly address all such cases, so these issues are prominent and are usually addresses manually or in a semi-manual manner. I bet there are plenty of plugins and scripts similar to Extrapass but perhaps better maintained. You're also encouraged to create one yourself, that addresses the issues you encounter.

Answer 2

I've had similar problems. I can offer an easy fix that does a partial job (it was good enough for my needs at the time).

A fairly simple fix to implement using IDAPython is applying the following logic:

1. iterate over all addresses
2. if addresses contains data and has a code xref pointing to it -> convert to code
3. retry defining functions adjacent to the correction you just made
4. if addresses contains code and has a data xref pointing to it -> convert to data

Keep in mind this is a general guideline to what you need to implement. The actual implementation requires some fiddling. This doesn't solve everything but it solves many of the problems. It helped me sort through a very troublesome IDB.

Hope this is what you were looking for.