I was reverse engineering a binary file. Details does not matter. The important part is that it is written in an architecture that is very unpopular, and there is no existing disassembler or decompiler that supports the architecture.

It's possible to write a disassembler without too much difficulty. Also, assume that I have the full documentation on how the machine code is executed.

However, writing a decompiler is much harder. So instead, I am thinking about using an existing decompiler and extend it to support my architecture.

Question:

  • What is a decompiler that can be extended to new architecture?
  • How can I extend the decompiler to support my architecture?
  • If you are looking for a decompiler with a GUI, try x64dbg. It's open source, available on github, so it should give you a nice starting point. – Twifty Oct 5 at 22:15
  • 1
    @Twifty x64dbg itself is not a decompiler, though a decompiler called snowman may be used as a plugin – SYS_V Oct 5 at 23:04

You did not mention what the format of the binary is. If the decompiler you choose to extend does not support the format of the binary you want to decompile then you will need to either implement support for that format or find a work-around.

There are a number of open-source decompiler projects under active development that you can choose from. Of course, there are also plenty of dead compiler projects, but they will not be discussed.

The projects discussed here are

  1. r2dec - a plugin for radare2 written in JavaScript

    • In the r2con2018 presentation "How to not write a decompiler" (video, slides), an overview of how the radare2 plugin r2dec works is given, as well as some instructions and advice on how to extend it to new architectures (beginning at 17:59 of the presentation). If radare2 (a disassembler) does not support the architecture you are interested in, then you will need to implement this yourself.
  2. reko - standalone toolchain implemented in C#

    • See the HowTo in the wiki for instructions on how to proceed with extending it to a new architecture
  3. RetDec - uses the Capstone disassembly framework and is based on LLVM
    • An overview of how it works is provided here: (slides, video). Discussion of disassembly and translation to IR begins at around 17:08 (slide 25). Extending it to an unsupported architecture probably will involve adding support to Capstone for disassembly as well as the asm -> IR translation in RetDec.

This is not meant to be an exhaustive list, of course. There are other options as well, such as radeco and snowman, but these do not seem to be very well documented.

If it is decided that decompilation is not required and disassembly will suffice, a nice guide to adding a new processor module to IDA is given in the following articles:

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.