3

I am trying to reverse engineer an executable that uses a lot of control flow flattening obfuscation (i.e, lots of subroutines that end with something like jmp esi, jmp eax) IDA Pro isn't able to resolve where the jump can branch off to and I'm not really sure what to do here.

I made a plugin for x64dbg ( https://github.com/JeremyWildsmith/x64dbg_scripts/blob/master/traceAreas.py ) that locates and monitors the branching instructions in a provided list of subroutines and generates graphml diagrams that I am reading (i.e example like this: https://i.imgur.com/kL4USza.png ) that I am viewing and analyzing in yEd live.

But it generates a lot of data. I know there is a vm hanging around somewhere in there and I am having trouble finding it just because of all the obfuscation. The collected control-flow data is here (warning, big text file, 45kb) https://github.com/JeremyWildsmith/x64dbg_scripts/blob/master/flow.graphml

When I open it in yEd Live it looks like there are multiple virtual machines or something? Yeah not really sure where to go with this, if anyone has any advice that would be awesome.

If I look at the control flow in a radial diagram ( https://imgur.com/a/6QPgZ ) I think I can identify the center as some sort of dispatcher or interpreter for the bytecode instructions? But what throws me off is how much branching goes on later on through the execution.

Thanks.

Improve this question
9
  • 1
    Have you considered trying Triton? Or any symbolic execution based analysis tool?
    – perror
    Mar 21, 2018 at 16:30
  • I have looked at Triton (and angr) but the code I am studying is unpacked and written to memory during execution and has a lot of anti-debugging techniques. So I'm not sure how to use the tool. Would it work to run it and them dump the executable and do symbolic execution on that? Then I don't have any state information.
    – Jeremy
    Mar 21, 2018 at 18:47
  • 1
    It mainly depends on the complexity of the packer. If it relies heavily on crypto and on low-level system calls, then you can forget about these tools. But, if this is only easy compression or encoding, then it is still possible (though the symbolic execution is not really efficient against packers).
    – perror
    Mar 21, 2018 at 18:55
  • @perror Do you know if there is there any way I can capture a state or something to that effect mid-way through execution and use it with angr? I'll run it through angr and see what I get anyways; but I'm skeptical how useful the output could be.
    – Jeremy
    Mar 21, 2018 at 18:58
  • Well, if you can create a core file at some point, I am quite sure that the binary loader of angr has an entry to load those files (see here). But, I never tried it...
    – perror
    Mar 21, 2018 at 19:05

1 Answer 1

Reset to default
1

One would have to look at the code. How are the jump targets calculated? From your diagrams it could also be something as simple as:

void jump_to_x_plus_5(uint x) { jmp x+5; }
void jump_to_x_plus_7(uint x) { jmp x+7; }
void jump_to_x_plus_9(uint x) { jmp x+9; }

And they just replaced all absolute jumps in the .asm with calls to random.choice() of those functions.

Improve this answer
6
  • Jump targets are calculated using heap data as input. I believe it may be possible to reverse engineer nut there are a lot of subroutines.
    – Jeremy
    Mar 21, 2018 at 17:52
  • @Jeremy okay, that does indeed smell like a vm. If it was just obfuscated jumps, you could have calculated the predicates with symbolic execution and patched in the original jumps. Now I fear you have to understand what’s going on. Maybe tracking memory accesses could give you an idea where the virtual IP is, etc. ?
    – manduca
    Mar 21, 2018 at 18:51
  • Hey Manduca, yeah its very possible to reverse engineer the algorithm from what I can see. I'll need to find a way to automate it though because there are over 60 subroutines easily where I'd have to do this. The control flow I expected was a loop and some branching but what i see is a ton of branching all over the place. So its either multiple vms or maybe I am missing something. This difficulty is finding out what the instructions do because they jump through a lot of obfuscated code.
    – Jeremy
    Mar 21, 2018 at 19:00
  • 1
    @Jeremy i just saw your conversation with perror. Weirdly enough a few days ago i wanted to dump a process in windows and then do some emulation on that state with unicorn. I ended up dumping as windows minidump (e.g. by right-clicking in process explorer) and then parsing that with github.com/utds3lab/sigpath/blob/master/scripts/minidump.py . Maybe you could do something similar for angr?
    – manduca
    Mar 21, 2018 at 19:29
  • Thanks for the advice, what does this minidump script do?
    – Jeremy
    Mar 23, 2018 at 7:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.