Most of file formats IDA handles are structured in a linear way, for example EXE or ELF or Java's Class file and so on.

However when it comes to tree-structured file format, I can't come up with a good way to properly handle them.
For example, recently I met a obfuscated PYC file, and the code has an OLLVM-like obfuscation plus some junk bytes, so a strong disassembler like IDA is needed. The PYC file format is in fact some magic byte + python marshal data, which is easy to parse. However, the marshaled data is in tree structure. Here's an example: http://www.beesfun.com/2017/04/01/PyCodeObject%E5%92%8Copcode/
As you can see in the parsed result, the root object is a module whose consts contain other functions in module, and the function in module can also have sub-functions in their consts. Also dis module or pytype can be used to parse the PYC file

Like Java's compiled file, PYC stores bytecodes and some metadata for each functions, functions call each other with name instead of address, bytecode believe the address starts from 0 in every function. So I studied the source code of module/java in idasdk73, trying to understand how the developers from HexRays cope with the Java class file format.
Then I found Java module in IDA are making a separate segment for each method and slot area. Because the python bytecode assume address starts from 0 in every function, and in every function there's a separate const pool which is also indexed from 0, I decided to create a header segment, a code segment and a const segment. But this same way won't work for a PYC file, as the PYC format is tree-structured (sub-functions are stored in parent const pools), and IDA does not allow overlapped segments. So I can't make the consts area a single segment.
But if I create a segment for each const, there will be tens of thousands of segments if the original py file is big.

Is there a better way to make a loader for the tree-structured format like PYC?


In not sure why you can’t just use several non overlapping segments but maybe the .pyc processor module from The IDA Pro book will be useful (I think it includes a loader)

  • I looked it but it seems that it does not have an loader? Seems that he didn't parse the header of pyc. And in fact my pyc processor is adapted from his one. For the reason why I don't choose to use the non-overlapping approach, that's because I want to preserve the reference to const area, however const areas contain functions, and those functions may contain smaller functions. I can't imagine how to write a loader for things like this :( – Misty Aug 18 '19 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.