Lifting up binaries of any arch into an intermediate language for static analysis

Question

Background

As the title is self-explanatory, I would like to translate binaries of any architecture (e.g. x86, ARM, ARM Thumb) to an intermediate language in order to apply arch-independent static analysis.

To be exact, my work is confined to the shared objects supplied in APK files for Android platform. My basic requirements, which I would expect the IL to meet, is as follows (Actually my goal is to extract information flows from a given .so file supplied in an APK file).

• Binary slicing
• PDG (CFG/DFG)
• Well-supported by its maintainer or its community

For this purpose I've looked into some existing tools listed below, but unfortunately I'm not sure whether I can use them to reveal information flows or not.

• OpenREIL: The aim of this project is to lift up arch-dependent binaries into REIL.
• Barf Project: A multiplatform open source Binary Analysis and Reverse engineering Framework
• Capstone: A disassembly framework.
• Epic: This tool translates binaries of any-arch to arch-independent LLVM bitcode. (This project is not public, so I cannot use it.)

Question

Is there any IL out there that I use to statically analyze a arch-dependent .so file (within an APK archive) in order to extract information flows? Basically I want it to provide basic requirements such as slicing and PDG.

Answer 1

My recommendation is to use libVEX, as it's the most robust intermediate code library that I know of. It's part of Valgrind and supports a load of architectures. You can either use libVEX in a C/C++ program or, alternatively, use the Python bindings called PyVex.

In any case: consider that most of the things you want to do must be implemented from scratch. Everything but translation of assembly "dialects" to an intermediate representation, I would say. Or, perhaps, you can use some parts of the Angr Project. I haven't tested it myself.

Good luck!

Answer 2

Radare2 may be what you are looking for:

https://github.com/radare/radare2

They lift everything to an intermediate language called ESIL so that it can be emulated easily. This code can be programmatically accessed, analyzed and driven through their libraries or through bindings in C, Python, etc. They have way more architectures and file formats than you even mention. You may be able to do everything you need with this.

Answer 3

I reviewed about 14 intermediate representations for the project I'm working on. It seems like any author (even for PhDs and master thesis) found all other existing IRs lacking and invented their own one.

There are two notable exceptions:

VEX is a prehistoric approach do IRs and provides a stable backend. That being said, it employs helper functions for stuff like flag calculations and thereby may omit semantical information.

REIL is well designed for the purpose of static analysis, but is fragmented ever since big G bought zynamícs. Some community projects keep the concept alive, but introduce their own extensions to REIL.

Since static analysis requires a SMT for most of its heavy lifting, we resorted to converting IRs to logic formulas and have been using them as a kind of intermediate representation.

For example:

pop eax

equals to:

esp = esp -4

[esp - 4] = eax

Answer 4

I would like to add BAP to your list. Disclaimer, I'm one of the authors. We just released the 1.0.0 version. This is our tenth public release (after about ten years of in-house development). Although our target audience consists of professional researchers and governmental agencies, we still have a rather decent community of independent researchers. We're also very responsive on a gitter channel, and our issue tracker.

Our intermediate representation has a formally specified semantics, and our lifters are intensively verified with our verification framework. To the day, our lifters support many architectures - x86, x86-64, mips32, mips64, powerpc32, powerpc64, and armv7.

It would be unfair not to mention other libraries, at least Radare2 and Angr.

Radare2's intermediate representation is called ESIL. It is not human readable (unless you're Forth programmer), but it is believed to be efficient. They have support for at least 5 architectures (ARM, x86, GameBoy and 8051).

Angr uses VEX library (that is GPL'ed) for the lifter, so they have out-of-the-box support for 5 architectures (x86,arm,mips,ppc,s390). The VEX language is also unreadable (although it is still better than ESIL) and loses some precision. We used VEX in previous incarnations of BAP but moved away to our own lifters many years ago.

Answer 5

I have some related researches on a few months ago. I don't know any tool enough all your requirements. However, I think you can follow Jakstab. It supports to analyze for Linux ELF files and generates control-flow graph. In generally, you need to modify Jakstab for your researches.