Can Machine Learning be used to develop better reverse engineering tools? [duplicate]

Question

So i googled this but other than some papers i couldnt find any reverse engineering tool that was built using machine learning

I'm not an expert in machine learning and deep learning but it seems rational to think that considering we have billions of open source codes out there, we can use them so our "machine" can learn how the assembly and executable of these codes look like and just study on them, mastering the art of reversing, and therefore building a tool this way that can reverse any given program with a great accuracy

Now is this doable or am i missing something here? is there any tool built this way or on its way to coming out? what are your thoughts on this? is better reversing tool even needed or is there already a great reversing tool that can do the job with the best accuracy possible?

Answer 1

If you consider genetic algorithms as a type of machine learning, here’s an interesting post on using it to create an assembler from opcodes and disassembler:

A Fast Track To Assembler Writing

You can also check Joxean Koret’s experience on using ML to match binaries to source code, including with machine learning:

https://docs.google.com/presentation/d/1ifvugStGL7Qc8xSFeYXp2MGQ6jQGOOMSolBrJy8kCMY/mobilepresent

https://github.com/joxeankoret/pigaios

Answer 2

we can use them so our "machine" can learn how the assembly and executable of these codes look like and just study on them, mastering the art of reversing, and therefore building a tool this way that can reverse any given program with a great accuracy

This is a pipe dream for several reasons:

1. Reliably differentiating between code and data in an executable binary is an undecidable problem. Data can be embedded in code, and the same sequence of bytes may be disassembled in different ways. Furthermore, code may be self-modifying. See

   • Why is disassembly not an exact science?
   • Soundness of ARM disassembly

2. Function start detection in stripped binaries is an undecidable problem.

   • Architecture Agnostic Function Detection in Binaries

3. Compilation is a lossy process, particularly when optimization takes place. Therefore decompilation cannot be fully automated and requires human decision making.

   • Certainly, full automatic decompilation of arbitrary machine-code programs is not possible, it is an undecidable problem like most problems of program transformation. So, from the point of view of theory no algorithm exists for decompilation of all possible programs. However, this (obvious) answer has no practical importance — many problems are undecidable or NP-complete yet there exist algorithms giving precise or approximate solutions for practical important cases. Thus the challenges of decompilation are in identification of subclasses of low-level programs and developing methods and algorithms targeted for these particular classes. The methods may even ask a human expert for help at key points. [ 1 ]

   • Certainly, fully automated decompilation of arbitrary machine-code programs is not possible -- this problem is theoretically equivalent to the Halting Problem, an undecidable problem in Computer Science. What this means is that automatic (no expert intervention) decompilation cannot be achieved for all possible programs that are ever written. Further, even if a certain degree of success is achieved, the automatically generated program will probably lack meaningful variable and function names as these are not normally stored in an executable file (except when stored for debugging purposes). [ 2 ]

   • Why can't native machine code be easily decompiled?

4. Machine learning - in this case, supervised learning - can simplistically be thought of as using algorithms to construct models from data, and the constructed model is then used for prediction or classification. When we say that an algorithm is is learning from data, it means that an algorithm is building a mathematical model by optimizing a loss function for the given data. There is no mind, no thought, no thinking machine. Algorithms do not look at anything, nor do they study anything.

---

1) C Decompilation: Is It Possible?

2) Is Decompilation Possible?