4

Please bare with me as I explain this situation.

Maybe 10 days ago I took up the challenge of reverse engineering an Android app. Learning from scratch, I installed ADB, Apktool, Android Studio, Notepad++ with Smali highlighting etc. My approach has been to write test programs in Android studio that mirror the workings of the APK, and then decompile with Apktool to help me do Smali modifications. I've been successful in modifying the application to log all HTTP requests, headers, cookies, and post data to the android log.

My next challenge was to figure out how an important algorithm in the app works. This is what stumped me. I spent the last 3 or 4 days spending the majority of my day analysing Smali code making almost no progress. Apparently the algorithm is done (at least in part if not mostly) in a native library with the .so extension.

One of the extremely frustrating things about Reverse engineering is how small the community is. There's very little resources on the web (At least compared to other things). I've probably bitten off more than I can chew. I always attempt difficult projects that are above my skill level. For this task I'm guessing I need to become very familiar with ARM and I'll have to use IDA Pro to analyse the .so file? To explain my knowledge level:

  • I have very little experience using Ollydbg in Windows (I have slight understanding of registers and CMP, JMP, ADD commands etc)
  • I have no experience using IDA Pro. I'm quite new to reverse engineering, but I've had success with simple Smali modding because it's simple in some situations.
  • I know basic Java (To be clear I'm not a complete beginner, variables, for loops, arrays, classes (to an extent) are second nature to me)
  • I know basic C++ if not more than basic
  • I've done a lot of Vb.NET programming.
  • I feel quite knowledgeable in Python.

So I'm experienced with programming but not really reverse engineering. How in over my head am I attempting to understand this complex native library (When I already know practically nothing about native libraries/JNI)? Could the professionals here please give me some specific examples of how I can get to the level where I am knowledgeable enough to complete my goal? I don't want to just give up because this is a difficult challenge. Please give me suggestions of how I can progress enough to complete my goal. I assume I'll need to learn IDA Pro and how ARM works.

Thanks

  • 1
    There is nearly nothing you have to know with IDA PRO if F5 key works on your SO file functions.. you could do CTRL+F5 and read code which if you understand programming language you would have not so much difficulty understanding the algorithm it pumps out. If you cannot do F5.. then yes.. you will have more difficulty.. since it's .so to my understanding this means linux.. I only did .DLL / .EXE with IDA PRO have no idea how .so files handle.. could be exactly the same just different operating systems. – SSpoke Jan 15 '16 at 8:56
  • If you have alot of time.. find out which export of the .so file you need and trace it in ollydbg.. I'm sure ollydbg is ported to linux and could load up .so files, so a artificial call of that export making a simple program that loads up that .so file and calls that export and trace it. If all else falls.. and you need answer to that algorithm what I usually do is make TCP/UDP servers for a specific .dll file call then retrieve the answers by sending specific packets to my server. – SSpoke Jan 15 '16 at 8:59
  • did you try both idaq? and idaq64? It should work.. I found this post indicating it's the same thing on windows and linux reverseengineering.stackexchange.com/questions/2664/… – SSpoke Jan 15 '16 at 9:01
  • Decompile to java, not to smali. E.g. dex2jar then JAD or luyten.Or Jeb :) PS IDA Pro has no problems with ARM code, at least the paid version. – Vitaly Osipov Jan 17 '16 at 8:14
  • @VitalyOsipov it seems that Enjarify is also good at converting a dex to jar. But it seems like programs that convert Jars to Java don't work 100%. It would be extremely nice to be able to step through the decompiled Java code with a debugger, but I tried and it didn't work for me. I followed this tutorial: blog.netspi.com/attacking-android-applications-with-debuggers But my debugger ended up underlining nearly all the Java in red :( – 43.52.4D. Jan 17 '16 at 20:29
7

I spent the last 3 or 4 days spending the majority of my day analysing Smali code making almost no progress.

I know it feels like you've made "no progress", but I'd encourage you to not look at it that way. You spent 3 or 4 days figuring out which approaches don't work, which is in itself progress. And you also built up 3 or 4 days of reverse engineering experience, regardless of how fruitful the immediate outcome was.

Apparently the algorithm is done (at least in part if not mostly) in a native library with the .so extension.

...

How in over my head am I attempting to understand this complex native library?

Given that you've determined that the Java code calls out to the .so library, it shouldn't be too difficult to find the native function in the .so library since the native function would be exported-by-name for JNI compatibility. This means that you don't in fact need to "understand this complex native library" in its entirety, but rather just the one native function in question (in addition to the functions that that function calls).

If the native code is not heavily obfuscated and if you have Hex-Rays for ARM, it should be relatively easy to understand the target function. If, on the other hand, you don't have Hex-Rays, then you can use the evaluation version of IDA Pro to disassemble the target function. You'd need to manually analyze the ARM instructions to determine what the function is doing. Although that can be tedious, ARM instructions and the architecture in general are very well-documented. Approach it as you would when learning any new programming language.

Start analyzing the function from the very first instruction, and keep high-level notes on what each instruction is doing in the context of the function. Keep track of what values get stored in what registers and how memory is used. On first-pass, your goal is to determine what the function is doing (based on the instructions). Once you've extrapolated what the function is doing, your second-pass should focus on trying to understand why the function is doing what it's doing. After some time, things should begin to "click" in your mind, you'll get that "aha!" moment, and you'll understand how the target algorithm works.

  • 3
    Nitpick: JNI functions don't have to be exported by name. They could be registered in an obfuscated JNI_OnLoad function. I've never seen that in the wild though. – Antimony Jan 15 '16 at 14:52
  • 3
    This is very true: "I know it feels like you've made "no progress", but I'd encourage you to not look at it that way. You spent 3 or 4 days figuring out which approaches don't work, which is in itself progress. And you also built up 3 or 4 days of reverse engineering experience, regardless of how fruitful the immediate outcome was." – Ta Thanh Dinh Jan 15 '16 at 15:38
  • 1
    @JasonGeffner well I realized the code was using a native library partly from searching online about others attempting to crack the same algorithm, and partly because there was a .so file in the lib folder matching the function names in Smali. Searching for the library name without the extension turns up zero results. But I think I know where it begins to be called. The difficult part for me now is because I have no idea how native libraries are called so I don't know what to search for. Knowing what parameters are passed to it, would be helpful. – 43.52.4D. Jan 15 '16 at 17:54
  • 1
    @JasonGeffner an interesting side approach is POSSIBLY just figuring out what parameters get passed to the function and then whenever I need to use the algorithm to generate a value, maybe I could just call the .so file without ever learning how the algorithm actually works. (Especially if it turns out to be a colossal set of instructions) – 43.52.4D. Jan 15 '16 at 17:56
  • 2
    If you don't have hex-rays, you might be able to use retdec as well, which produces some reasonably high quality output (relative to typical output of native code decompilers). retdec.com – broadway Jan 15 '16 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.