19

I'm analyzing some software that appears to encrypt its communications over the network, but it does not appear to be SSL. How can I easily determine what encryption algorithm its using, and maybe find the key?

  • Just to make it clear. Do you want to identify the encryption algorithm based on the assembly code of the encryption algorithm or based only on the encrypted packets ? – perror Apr 2 '13 at 17:00
  • @perror I have access to the binaries and its resulting network traffic. So which ever is easier. Using the IDA plugins suggested fixed my immediate problem, but is there an easier way, for future reference, to do it based solely on network traffic? I would have assumed that since the traffic had no header or identifiable information that it would not be possible. – omghai2u Apr 2 '13 at 17:09
14

Maybe check out this IDA plugin.

After you locate the crypto functions, doing a cross-reference in IDA should allow you to see where the functions are called and likely the key is nearby. If you can set a break-point on those functions and see what is being passed in for the key, this, of course, would be the easiest way.

9

For a bit more advanced way of automatic crypto identification see Felix Gröbert's work on Automatic Identification of Cryptographic Primitives in Software . He uses a pintool to dynamically instrument the code which can allow to even recover keys. The code is also available. The repository contains other tools used in comparison , such as PeID and OllyDBG plugins.

7

I have not used it but there is an open source tool called Aligot that may help when the encryption algorithms have been obfuscated. According to its authors, Aligot can idenfity TEA, MD5, RC4 and AES.

Aligot does have an important disclaimer:

Aligot was build as a proof-of-concept to illustrate the principles described in the associated paper. In particular it is not currently suitable to automatically analyze large programs. If you are interested in such project, please contact the author ;)

Despite the disclaimer, the results indicated in the paper suggest that Aligot is worth looking into.

5

A nice combination of findcrypt2 by HexRays and the work done by Felix Gröbert is IDAScope. It's very useful for searching for and identifying encryption algorithms. For more information on IDAScope's Crypto Identification I'd recommend the following link.

  • How does it improve on findcrypt2 and HexRays? What makes it better, faster, etc.? – Lizz Apr 6 '13 at 4:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.