5

In the past for years I used some disassembler and tried some decompiler, nowadays there's so much talking and stuff about deep learning and AI, I wonder if some can be used with those tasks (given some human training) and if there's some tool using it already.

closed as primarily opinion-based by NirIzr, peter ferrie, Megabeets, 0xC0000022L Jul 26 '18 at 11:06

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • I like this question and actually thought about it lately. While it is somewhat relevant to RE, I think ai.stackexchange.com may be more relevant since this issue is more about AI target definition and training than the actual process it does to disassembly/de-compile a program. – Elian Kamal Jul 22 '18 at 13:54
  • 1
    The mechanical process of disassembling – purely decoding instructions – stays the same; there is nothing there that an AI could do better. AI could help in determining what to disassemble – i.e., the old problem of differentiating between code and data, and then only for paths that are not clearly either a part, or not a part of executable code. Both linear-sweep and recursive disassembly can mark huge swaths of 'actual code' and '(likely) not code at all', but the gray areas in between are where the problems lie. ..(ctd) – usr2564301 Jul 22 '18 at 15:40
  • 1
    .. I am not convinced an AI would be able to tell us more than a pure mechanical process. I.e., the best possible disassembler should be able to "run" an entire program and inspect all possible code pathways, which is a pure mechanical exercise. The only reason one does not do that (yet) is because it's fundamentally an NP problem. Also, if all code was written 100% correct, you'd never experience a crash. – usr2564301 Jul 22 '18 at 15:42
  • @usr2564301 I think observing disassembled code there's a lot of pattern work, there are nice tools nowadays to graph where a function branch, but I think an AI would be helpful guessing/suggesting what a function does, what kind of data you're inspecting – Alex Jul 22 '18 at 16:53
5

First, you cannot answer to this question without having to define what you mean by AI... Because this is probably the worst name for a domain in computer science. Where people think about "programs mimicking human reasoning", the reality is more about "automated heuristics to recognize patterns in tons of samples"...

So, I will take the more accurate definition of AI, it is to say that I assume that an AI program can recognize patterns that might escape to the understanding of human being and that they can self-train to achieve this.

Now, speaking about disassembly, the core problems are definitely not about recognizing patterns but rebuilding all the execution paths that one can find in a binary. So, an AI algorithm is simply not interesting to choose if an execution path is more valuable than another because they are all interesting (as the disassembler need to exhaustively find all the paths).

Finally, speaking about decompilation, this is the field where you may want to use AI to recognize some patterns such as:

  • What compiler or what language has been used;
  • Guess if this code is similar to another sample that is in your data-base of malware;
  • Give a first guess of the architecture of the software (complex data-structures (arrays, records), functions, objects, modules, packages, ...) even after an obfuscation pass;
  • Recognize the most common functions and what they may be used for;
  • ... probably many other guesses that can be automatized ...

But, the point here is that we are just all stuck at the disassembly problem (which is a quite strong problem!!!) and to go further we first need to have a reasonably reliable method/algorithm/tool in order to reach the next level (decompilation).

Though, a few people have worked in this direction, you may look at (this list is far from being complete, I search only 10mn on Google):

4

Although I initially voted to close this question as primarily opinion-based, given both answers side with the same general answer ("No!") I'll answer as well. I just love being the devil's advocate.

A general stance

This question is very difficult to answer. As someone who came from the security field and had worked on quite a few machine learning related projects in the past, I know it's really hard to make an honest projection on whether it's possible to solve problems in our domain using machine learning. Often times ML/AI research is a lot about trial and error.

AI disassembler

Although the operation of disassembling a sequence of bytes is quite straight-forward, there are quite a few problems that are solved nowadays with somewhat clever heuristics. Those problems include:

  1. Identifying function boundaries.
  2. Classifying code VS data.
  3. Inferring structures and higher-level constructs from assembly.

and more.

As time went on, some of those problems have been improved mostly by creating and improving manually written heuristics and logical guidelines for disassemblers. Mostly based on human experience and incorrect results seen in the past.

However, it is still quite common to see errors in fully automated disassemblers and as evidence to that we can see the focus given by disassemblers to the reverse engineer to manually correct and adjust the automated analyses. One might argue it's possible that ML/AI approaches could create similar or better heuristics and more complex / accurate rules.

AI decompiler

Decompilation is generally seen as more difficult to accomplish even given perfect disassembly. It relies heavily on inferring higher-level constructs such as structures, objects, classes and inheritance. It requires more accurate data-type recognition and more demanding understanding and reasoning of abstract objects. Even humans find those harder to manually recognize, not to mention using automated rules and heuristics. Therefore, decompilation may prove a more fertile ground for such improvements.

A word about the buzzword

Nowadays, when startups rise and fall based on using the right buzzword in their marketing it is easy to get drawn into using such buzzwords instead of accurate descriptions and forcing the solution on the problem instead of finding the best solution for the problem at hand.

3

No!

Reason is that AI is dumb, much more than you think it is. All it can do is matching patterns to predetermine output. And here are two main problem to use it to decompile code or to recreate "readable" source code:

  • This matching isn't perfect. For example if you have mov, eax 0x10 it could be interpreted as i=8; as AI didn't have this particulate training set, but it has mov, eax 0x8.

  • Other big problem is what to feed AI. In translation tools are used sentences, in image recognition are used features from whole images. What to use in code? Few op codes? How many? where to start?

In conclusion AI (at this stage and near future) is mostly useless for this purpose.

0

I want to say yes, but I have to say NO. The thing is that computers grab our programs that we write and optimize them in a way that often will make little or no sense to humans. You'll see things being multiplied and arrays being worked with that you can understand what is happening , but it is very unhuman to work with data in this kind of way. Computers and humans think very differently.

I think could be possible for an AI to do the following:

  • Grab the assembly and convert and either make it clearer for humans to understand.
  • Make a near useable form of assembly. (Maybe.... Because we would be keeping the computer's understanding of the relative data intact. )

Take this simple example:

MOV v7, DWORD PTR [v7 + 0x8]

would convert to

v7 = *(_DWORD *)(v7 + 8);

then to

v7 = *(v7 + 8)

And this might be really something completely different in source.

where as you might see something like

mov eax, dword ptr ds: [ESI*deadbeef+0b0] 

and the computer will think

int a = somefoovar[v5].someint

or

int a = 1234;

Both are kind of correct... If that array doesn't have changing data in it.

but a computer might look at this as a single variable, when it could be something temporary. Also I have noticed that in my own decompilation work, that you'll wind up with a lot more static variables that are when you originally start.

I think some other problems might be that the program might not ever hit certain parts of a function and might not understand all event paths.

Personally, I would like to see AI what AI can come up with . Maybe it would be nice to have a pseudo translator of code. But I know that I will have to clean up after it and correct it's understandings.

Not the answer you're looking for? Browse other questions tagged or ask your own question.