1

I am trying to setup a TLS proxy with Charles proxy to man-in-the-middle an Android app. I would like to be able to view the contents of HTTPS traffic. This is not an app that I built, rather I am trying to reverse engineer an apk that I obtained. I have performed my usual static analysis of the apk, but I also want to dynamically analyze the HTTPS traffic. I am running an emulated Pixel 2 device with API 23 (Marshmallow).

I have read all the documentation I can find regarding how to use the Charles proxy as an TLS proxy for Android. However, much of the documentation is directed at setting up a TLS proxy for an app for which I have control over the AndroidManifest.xml and can thus set the network security configuration to trust user CAs--the default/documented procedure to setup a Charles TLS proxy involves downloading the Charles CA cert to the Android device and installing it as a user CA.

I assumed that if I could force the charles CA cert into the System cert folder (as opposed to the user certs) then the TLS proxy would "just work." However, even after I copied the Charles CA cert into the "/system/etc/security/cacerts" directory I still can not MiTM HTTPS traffic for apps. In addition, when I try to MiTM web browser traffic, the browser still claims there is an issues with the TLS connection (even thought the Charles cert is in the cacerts folder and should now be trusted). I confirmed the Charles cert is listed in the list of "System" CA certificate (although the settings app does crash if I try to view the cert metadata info--because there is none for that cert, just the encoded certificate).

Why is the Charles CA cert not trusted even when I have forced it into the system CA cert folder? Is there a better tool to be using besides Charles? Any help would be greatly appreciated as I am coming to my wits' end.

Improve this question
5
  • Particularly in the browser you could happen to run across public key pinning (sometimes called certificate pinning). I think for certain sites certain browsers (namely Chromium-based ones) even have built-in expectations for certain certificates (or at least stuff like particular issuer or particular fingerprint of the issuer or so ...). Other TLS-based communication may not be as well protected. I have nothing to offer regarding the use of Charles, though.
    – 0xC0000022L ♦
    Commented Apr 20, 2020 at 20:56
  • Thanks for the comment. I don't think the app of interest uses public key pinning. I'm going to try proceed by trying a different TLS proxy and I'm also going to try proxying an app that I know for sure doesn't use public key pinning as a test.
    – hft
    Commented Apr 20, 2020 at 23:21
  • 1
    Try the following tool for certificate pining bypass: frida.re/docs/home
    – Sevastyan Savanyuk
    Commented Apr 22, 2020 at 6:28
  • Thanks for the suggestion. I was able to get what I needed using Burp Suite Community Edition. I'll check out Frida for future work.
    – hft
    Commented Apr 22, 2020 at 6:57
  • related question: security.stackexchange.com/questions/192378/…
    – Robert
    Commented Apr 22, 2020 at 12:46

1 Answer 1

Reset to default
1

I will answer my own question because I figured out what was wrong.

The app was not functioning correctly for a number of reasons that frustrated the analysis. For one, there were many advertising domains being contacted by the app on startup. The app made these connections over TLS and the connections would not succeed without trusting the proxy root CA certificate.

I got around the problem by using Burp Suite. This tool will automatically identify TLS requests that fail due to the proxy and will stop proxying those domains. Using burp suite I was able to see that all the failed TLS connections were advertising domains that I did not care about.

In fact, other than the third party advertisers, the app actually only uses HTTP and TCP to communicate with various servers. Thus I was able to get all the info I needed from the plain text HTTP requests and some other TCP connections that I monitored using Wireshark.

Update: As mentioned in the comments, for some reason some of the domains (the domains owned by the app maker) did not have any issue with TLS connections either because they accepted the Burp root CA or because those connections ignore TLS issues with the certificate (I'm not sure which). Most of the connections made by the app are HTTP, but I was also able to see HTTPS requests to the app's domain as mentioned above. I was not able to view the ad-related TLS traffic, probably due to public key pinning, but I am not sure about that.

Improve this answer
3
  • IMHO your "workaround" does not answer your question. You wer asking I would like to be able to view the contents of HTTPS traffic. Using this workaround most of the TLS/HTTPS traffic will just be bypassed without having access to the HTTP data.
    – Robert
    Commented Apr 22, 2020 at 12:52
  • Interestingly, I was able to view the HTTPS traffic I was interested in. The majority of request to servers owned by the app makers were HTTP, but some of the requests were HTTPS, which I could also see--for some reason those domains were not given exceptions by Burp Suite--they seem to either accept the Burp root CA or not care about whether their domain's certificate was valid or not (I'm not sure which).
    – hft
    Commented Apr 22, 2020 at 19:32
  • I don't know the reason the advertising modules don't accept the cert, but it may be due to public key pinning as mentioned in the other comments. Unfortunately, I don't have time to reverse engineering all the ad libraries...
    – hft
    Commented Apr 22, 2020 at 19:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.