10

Background

As the title is self-explanatory, I would like to translate binaries of any architecture (e.g. x86, ARM, ARM Thumb) to an intermediate language in order to apply arch-independent static analysis.

To be exact, my work is confined to the shared objects supplied in APK files for Android platform. My basic requirements, which I would expect the IL to meet, is as follows (Actually my goal is to extract information flows from a given .so file supplied in an APK file).

  • Binary slicing
  • PDG (CFG/DFG)
  • Well-supported by its maintainer or its community

For this purpose I've looked into some existing tools listed below, but unfortunately I'm not sure whether I can use them to reveal information flows or not.

  • OpenREIL: The aim of this project is to lift up arch-dependent binaries into REIL.
  • Barf Project: A multiplatform open source Binary Analysis and Reverse engineering Framework
  • Capstone: A disassembly framework.
  • Epic: This tool translates binaries of any-arch to arch-independent LLVM bitcode. (This project is not public, so I cannot use it.)

Question

Is there any IL out there that I use to statically analyze a arch-dependent .so file (within an APK archive) in order to extract information flows? Basically I want it to provide basic requirements such as slicing and PDG.

3

5 Answers 5

5

My recommendation is to use libVEX, as it's the most robust intermediate code library that I know of. It's part of Valgrind and supports a load of architectures. You can either use libVEX in a C/C++ program or, alternatively, use the Python bindings called PyVex.

In any case: consider that most of the things you want to do must be implemented from scratch. Everything but translation of assembly "dialects" to an intermediate representation, I would say. Or, perhaps, you can use some parts of the Angr Project. I haven't tested it myself.

Good luck!

5
  • does VEX handle flag calculations now? They omit semantics information, rendering it infeasible for some static analysis use cases
    – Nordwald
    Sep 28, 2016 at 7:19
  • Yes, it does perform flag calculations and calls internal checking functions to determine if according to the translated flags the jump should be taken or not. Oct 3, 2016 at 7:39
  • Sadly, vex still can not express all flag calculation in its own syntax still lazy helper functions. :/ You won't gain semantic information from it unless you parse them all by yourself.
    – Nordwald
    Oct 4, 2016 at 5:11
  • Ah, yes, that's true. However, I fail to understand why it can cause big problems. Could you please give me an example where the VEX calculated flags + manual calculations can be problematic? Or is it just that it's annoying because you have to re-invent the wheel? Oct 4, 2016 at 10:28
  • 1
    Its more about reinventing the wheel. VEX does internally resolve its helper functions. Most projects using this information (like angr) end up re-implementing all helper functions.
    – Nordwald
    Oct 4, 2016 at 11:26
4

Radare2 may be what you are looking for:

https://github.com/radare/radare2

They lift everything to an intermediate language called ESIL so that it can be emulated easily. This code can be programmatically accessed, analyzed and driven through their libraries or through bindings in C, Python, etc. They have way more architectures and file formats than you even mention. You may be able to do everything you need with this.

4

I reviewed about 14 intermediate representations for the project I'm working on. It seems like any author (even for PhDs and master thesis) found all other existing IRs lacking and invented their own one.

There are two notable exceptions:

VEX is a prehistoric approach do IRs and provides a stable backend. That being said, it employs helper functions for stuff like flag calculations and thereby may omit semantical information.

REIL is well designed for the purpose of static analysis, but is fragmented ever since big G bought zynamícs. Some community projects keep the concept alive, but introduce their own extensions to REIL.

Since static analysis requires a SMT for most of its heavy lifting, we resorted to converting IRs to logic formulas and have been using them as a kind of intermediate representation.

For example:

pop eax

equals to:

esp = esp -4

[esp - 4] = eax

5
  • Hey Nordwald, may I ask whether you employed REIL eventually? If so, how many additional efforts need to be performed in order to get it work for symbolic execution (since you mentioned SMT solver)? thank you! Mar 1, 2017 at 16:19
  • A range of REIL implementations like BARF already contain translation to the statement solver. Just have a look at the examples here: github.com/programa-stic/barf-project/tree/master/examples/…
    – Nordwald
    Mar 1, 2017 at 16:23
  • That's great! Thank you a lot for the resource. Mar 1, 2017 at 20:22
  • BTW: may I ask what "front end" you recommend to use? I have the newest IDA-Pro + Binnavi... Mar 1, 2017 at 20:23
  • Hey, we are using an in-house-solutions here since a proper cfg is crucial for data flow analysis, but you may be fine using IDA. I recommend keeping the heavy lifting away from it (e.g. make an export plugin).
    – Nordwald
    Mar 2, 2017 at 6:07
4

I would like to add BAP to your list. Disclaimer, I'm one of the authors. We just released the 1.0.0 version. This is our tenth public release (after about ten years of in-house development). Although our target audience consists of professional researchers and governmental agencies, we still have a rather decent community of independent researchers. We're also very responsive on a gitter channel, and our issue tracker.

Our intermediate representation has a formally specified semantics, and our lifters are intensively verified with our verification framework. To the day, our lifters support many architectures - x86, x86-64, mips32, mips64, powerpc32, powerpc64, and armv7.

It would be unfair not to mention other libraries, at least Radare2 and Angr.

Radare2's intermediate representation is called ESIL. It is not human readable (unless you're Forth programmer), but it is believed to be efficient. They have support for at least 5 architectures (ARM, x86, GameBoy and 8051).

Angr uses VEX library (that is GPL'ed) for the lifter, so they have out-of-the-box support for 5 architectures (x86,arm,mips,ppc,s390). The VEX language is also unreadable (although it is still better than ESIL) and loses some precision. We used VEX in previous incarnations of BAP but moved away to our own lifters many years ago.

1

I have some related researches on a few months ago. I don't know any tool enough all your requirements. However, I think you can follow Jakstab. It supports to analyze for Linux ELF files and generates control-flow graph. In generally, you need to modify Jakstab for your researches.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.