Recently on Reddit ReverseEngineering I stumbled on a self-modifying code in Python. Looking at the Github repository was quite instructive and I found picture of the Python bytecode program exposed in CFG form:
I am wondering if there are tools to perform static analysis on Python bytecode program with some nice features (such as generating the CFG or allowing to manipulate the code, ...) ?
There are several tools dedicated to Python's bytecode reversing:
Uncompyle and Uncompyle2
'uncompyle' converts Python byte-code back into equivalent Python source. It accepts byte-code from Python version 2.7 only. The generated source is very readable: docstrings, lists, tuples and hashes get pretty-printed.
'uncompyle' may also verify the equivalence of the generated source by by compiling it and comparing both byte-codes. 'uncompyle' is based on John Aycock's generic small languages compiler 'spark' and his prior work on 'decompyle'.
pyREtic, that is more a powerful framework than a simple program
pyREtic is an extensible framework to assist in performing various reverse engineering tasks for Python language projects. It assists a reverse engineer in gaining sourcecode (.py's) back from bytecode (.pyc's), in particular it assists when the code that is being reversed has put some effort into trying to stop decompilation using standard toolsets.
pycdc, which works better than uncompyle, and is simpler to use than pyRetic
Decompyle++ aims to translate compiled Python byte-code back into valid and human-readable Python source code. While other projects have achieved this with varied success, Decompyle++ is unique in that it seeks to support byte-code from any version of Python.
Decompyle++ includes both a byte-code disassembler (pycdas) and a decompiler (pycdc).
The Manyard framework, which is dedicated to Python3
Maynard is a (dis)assembler for Python byte code written by a member of Python core and the release manager for Python 3.4. Reading material here and here. I'm not aware of a public tool (besides the one you linked) that can do CFG visualization like that, but you certainly could build one on top of Maynard.
pyREtic from Immunity Sec can also provide some help in looking into original source code and perform modifications as well.
You may be interested in review the capabilities of the tool in this document: "pyREtic, In memory reverse engineering for obfuscated Python bytecode" by Rich Smith [PDF]
A challenge in writing a tool to extract the control flow of python bytecode is that there are so many Python bytecodes versions to choose from, about 25 or so by now (if you include pypy variants).
The bytecode in the example graph with its JUMP_IF_FALSE followed by some POP_TOPs and PRINT_NEWLINE instruction, reflect Python before 2.7.
However the example in one of the comments from the Flare_bytecode_graph with its POP_TOP_IF_FALSE is 2.7. Python 3 drops the PRINT_ITEM instruction.
Anyone writing such a tool will have to come to grips with that; or be happy with living in a single version of Python, for which 2.7 is probably the most popular choice. Or you could ensure that the version of Python you are running matches the bytecode you want to analyze and use the current dis and opcode modules that Python provides. But even here those modules change over time, not in the least being the particular bytecode instructions.
I wrote a python package called xdis for those who want to work across all versions of Python bytecode, and don't want to be tied with using the same version of Python that the bytecode requires.
The next thing that you'll probably want to do in this endeavor is to classify instructions into categories like those which can branch and those that can't and if the branch is conditional or not.
Python has some lists that cover some of this, ("hasjrel", "hasjabs") but alas it doesn't have the categories that are most useful. And for possibly historical reasons the categories are lists rather than sets. But again xdis to the rescue here; it fills this information in with sets "CONDITION_OPS", "JUMP_UNCONDITIONAL" and "JUMP_OPS".
Using this I've written https://github.com/rocky/python-control-flow which uses xdis and has some rudimentary code that will create a control flow graph and dominator tree for most Python bytecodes. There is some code to create dot files and I use graphviz to display that. I notice that Python, can create a lot of dead code.
The intended use of that package is to reconstruct high-level Python control structures. There is some rudimentary control structure detection, although this needs a lot more work. Python control structures are pretty rich when you include the try/while/for "else" clauses, and the "finally" clauses. Even as is, the annotated control flow of the basic blocks very helpful in manually reconstructing structured control flow.
When this is finished, I can replace a lot of the hacky code for doing that in uncompyle6.
And this leads me to the list of decompilers mentioned in the accepted answer...
As stated, those particular versions of uncompyle and uncompyle2 handle only Python 2.7. As suggested, there are older versions that handle multiple Python versions 1.5 to 2.3 or 2.4 or so. That is if you have a Python 2.3 or 2.4 interpreter to run this on.
But none of these projects are actively maintained. In uncompyle, there are currently 25 or so issues with the code, many that I have fixed in uncompyle6. And this is for a version of Python that no longer changes! (To be fair though there are some bugs in uncompyle6 that don't exist in uncompyle2. And to address those, I'd really need to put in place that better control flow analysis)
A number of the bugs in uncompyle could easily be fixed by just doing the same thing that uncompyle6 does, and I think some of that I've noted in the issues. At this point uncompyle2 is much better than uncompyle, and if you are only interested in Python 2.7, that is the most accurate.
As for pycdc, while it is pretty good for its relatively small size (compared to uncompyle6), it too isn't maintained to the level that it would need to keep up with Python changes. So it is weak around Python 3.4 or so and gets progressively weaker for later versions. Uncompyle6 is like that too, but currently less so. pycdc has over 60 issues logged against it and I doubt those will be addressed anytime soon. Some of them aren't that difficult to fix. My own (possibly slanted) comparison of those two decompilers is https://github.com/rocky/python-uncompyle6/wiki/pycdc-compared-with-uncompyle6
The Flare-bytecode graph project can help with graphical bytecode CFG representation. Taken from the project README:
...
It is also possible to create control flow diagrams using GraphViz. The disassembly within the graph can include the output from a simple peephole decompiler.import bytecode_graph def Sample(): i = 2 + 2 if i == 4: print "2 + 2 = %d" % i else: print "oops" bcg = bytecode_graph.BytecodeGraph(Sample.__code__) graph = bytecode_graph.Render(bcg, Sample.__code__).dot() graph.write_png('example_graph.png')
