I'm working on one Android app for potential malware analysis and I am trying to MITM it. I found out that the verify function never executes (X509 class) when i hook onto my proxy but rather that app fails upon reaching this function
SSLEngineResult unwrap = this.f13022c.f13031h.unwrap(byteBuffer, a2);
This object is part of javax.net.ssl.SSLEngineResult
The exception that this function triggers is javax.net.ssl.SSLHandshakeException: Handshake failed
. I had feeling that they check system proxy and if its present that they mess with byteBuffer but that wasn't the case. I attached proxy without serving self signed certificate and app works normally, just the moment i attach self signed certificate trough Fiddler, it crashes on this function and exception occurs. It seems like this entire proccess of handshaking is causing problems and i don't know why. Could it be that the app devs did this on purpose somehow on lower level or could it be that this function itself is not working properly with my certificate (which should be the case really)