I am a newbie to RE and IDA Pro 7.2, and I am working my way through the book Practical Malware Analysis. I'm currently working on the lab 15-01 problem in the anti-disassembler chapter, but am having difficulty reproducing part of the solution in my environment.

I solved the lab by redefining some bogus opcodes into data, and now I have reasonable disassembly listings to look at. So far, so good. But at the end, the solution tells you to turn this fixed-up section into a function using the "P" hotkey. That doesn't work for me: I get this message in the output window:

.text:00401010: The function has undefined instruction/data at the specified address.
Your request has been put in the autoanalysis queue.

The address points to the first data definition I made.

Trying to flip over to graph mode doesn't work either. I get a popup that tells me that only instructions belonging to functions can be displayed in graph mode.

If I go to the Problems tab, I see lines like this:

.text:00401062  NONAME      db 0E8h

But I didn't glean anything in the IDA Pro help file for no name problems that helped me with my specific situation.

Ultimately, I used Edit -> Patch Program -> change byte to NOP out the data bytes and turn them into code. Then I had a function that I could graph.

My question is, is there a way to do this without patching bytes?

EDIT: Added error message text to the post.


I think you may face what's called a rogue-byte.

I'm currently scripting something to patch them (see IDAPython - Find the start of an undefined code blob).

Can you tell me if it's the same type of code logic?

If yes, you indeed might want to replace the rogue-byte with a NOP. You can't do it without patching the binary, because as you said, it's an anti-disassembly technique. It's here in order to mess with IDA Pro!

If you want to patch it:

  • select the line containing the rogue-byte, then undefine it (shortcut: "U", or go to "Edit -> Undefine")
  • select the instruction ("db 0E8h") then go to "Edit -> Patch program -> Assemble". You can now replace your rogue-byte by a NOP, by typing "nop", then "OK".
  • You now need to define the NOP as code (so it can get interpreted as an instruction), by pressing the C shortcut, or by going to "Edit -> Code".
  • Finally, you can define your function as a proper function, by reaching the start of it (loc_XXXXX for instance) and pressing the P shortcut, or "Edit -> Function -> Create function".

If you want, I can share you the IDAPython script that I wrote, even if it's still not complete yet.

| improve this answer | |
  • Hi Guillaume, thanks for reply. It sounds like your script will be very helpful for automating the process of dealing with this technique. I hope you'll update this thread with a link when it's ready to go. – Polly Aug 5 '19 at 0:21
  • IDA is simply buggy; it doesn't understand that code may contain embedded data in many popular CPUs; any data embedded in code becomes "anti-debugging" even it is normal Assembler code. Pathching such embedded data with NOP is not always possible, because the code may need the data. Of course, it is possible to patch it, create functions and unpatch later, however, the idea look ugly. – Vitalii Oct 23 '19 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.