I'm trying to RE a Windows executable compiled with VS 2008. After the initial autoanalysis most functions are detected correctly; however, some have wrong end address — for some reason IDA places the end of the function earlier, leaving a chunk of code not associated with any function. The undetected function gets noreturn attribute and often has sp-based autoanalysis failed message. I have fixed most of these problems by hand. The problem is that Hex-Rays seems to use autodetected function boundaries and therefore the decompilation fails after a jump inside the function, which looks like a jump outside to Hex-Rays.

Example hex-rays output:

void __thiscall sub_403CE0(void *this, unsigned int a2, int a3)
    sub_407F30(this, a2, a3);

As you can see, there is a JUMPOUT statement at the end of the function. This was correct before the function boundaries were adjusted, but now locret_403CFD belongs to the function itself and is not a jump “out”. Is there a way to tell hex-rays the function doesn't end here and pass a correct function start/end addresses to it?


Yes there is. HexRays takes function boundaries from IDA disassembly, so as far as you can change the function boundaries in disassembly view, you can manage function boundaries as follows (the solution is for you specific case, solving this problem in common may be more complicated):

  1. Find the function in the disassembly window
  2. Go to the end of the code not associated with any function but should be in function you are workung with, press your cursor there.
  3. Press E. This will mark the current position as an end of the function before the area
  4. Go back to decompiler window. Press F5. JUMPOUT should disappear if the address it jumps to now belongs to the function you are working with.
  • 2
    This is almost exactly what I did (didn't know about the E though — thank you for that — so I undefined the function and defined it again). Unfortunately, the function boundaries are correct and JUMPOUT doesn't disappear. This looks like a problem with this particular database because it doesn't reproduce on a copy of the same binary, but I would prefer not to start it over — several hours of work would be wasted. – John Doe Sep 20 '15 at 14:28
  • Sorry for dumb question - did you press F5 in decompiler window after function redefinition to refresh the view ? If it doesn't help, try the following: 1 - Close decompiler window and delete the function (Edit-->Functions-->Delete function, you shlould stay in disassembly window in the function boundaries) 2 - redefine the function again (select all its instructions, press P) 3 - press F5 again. Good luck anyway. – w s Sep 20 '15 at 14:41
  • Yes, I did. I have also tried deleting the function. Now I'm trying to write a plugin that calls mark_cfunc_dirty function. – John Doe Sep 20 '15 at 15:09
  • Unfortunately, even calling clear_cached_cfuncs doesn't help. I'll try something else, but my best guess is that I'll have to start over with a clean database. – John Doe Sep 20 '15 at 17:21

After some work with the database I figured out the reason of the strange behavior of IDA. The binary I'm trying to decompile is a GUI application that uses MFC; I have imported a number of MFC types from a PDB file obtained from Microsoft Symbol store. It's either an IDA bug or corrupted PDB — if I don't import types, all functions are detected correctly and decompilation works as expected, if I do — many function boundaries go wrong, decompilation fails, function attributes are ignored by Hex-Rays and so on.

One of possible workarounds is to export the database as IDC file and then run the resulting script on a clean database. This way, function prototypes and type information are kept intact. Hex-Rays decompiled results will differ, though (variable mapping information is lost, and so are local variable names).

  • Perhaps IDA is trying to apply the function boundaries of the MFC DLL to your EXE? – SamB Mar 15 '16 at 23:45
  • Unfortunately, I have this issue with a very simple program not using MFC... no matter what I try, pseudocode will always consist of a JUMPOUT... – Ray Koopa Feb 27 '17 at 19:58
  • @RayKoopa did you import any types? Are you sure the actual code wasn't written in that way? – John Doe Feb 28 '17 at 8:23
  • It is actually the main function of a packed program, nothing done, a fresh database. There's a CALL to a location just skipping over the CC debugger trap following it. The location however is part of the main function calling it. – Ray Koopa Feb 28 '17 at 11:01
  • Decompiling a packed program is unlikely to produce any meaningful results. The unpacker code was likely written in assembly and/or purposefully mangled. – John Doe Feb 28 '17 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.