How much Cryptography knowledge is important for reverse engineering? [closed]

Question

Is understanding of Cryptography really important for a reverse engineer?

Thanks.

Answer 1

It is more and more important for practical reverse-engineering. It is now present in malware, the example of Stuxnet, Flame and others are quite typical of the usage of cryptography in such context. And, it is also present in most protection schemes because a lot of techniques use cryptography to protect the code and data. Just consider software such as Skype or iTunes which are relying on cryptography to protect their protocol or to hide information in the executable.

So, indeed, it would be really a problem if you do not understand a bit cryptography when reversing. And, by "understanding cryptography", I mean at least to be able to recognize the code of classical cipher algorithms at assembly level such as DES, AES, SHA-1, SHA-3, and so on. And, also to know classical flaws and cryptanalysis techniques for weak crypto (such as frequency analysis).

A good way to learn about the cryptography needed for reverse-engineering would be to implement (with the help of existing codes found on the Net) your own cryptographic library with classical ciphers and look at the generated assembly. If you do not have the patience to do so, just look at the crypto-lib of OpenSSL, get it compiled and look at the code and the assembly.

Of course, more you know about it, more you will be efficient when facing it.

Answer 2

I'd say it's a very useful tool to have in the arsenal, but your use of it will depend upon your ultimate goals. Personally, I use it a fair bit on binary application assessments, but that's not the case for everyone. As I said, it largely depends on what you want to be doing.

At the end of the day, you will see cryptography being used in applications if you end up doing reverse engineering work. Understanding and breaking that cryptography may or may not be part of your job, but it's always nice to have at least a basic understanding of the field. It's always a nice +1 on your resume, too.

In terms of what you should learn, I'd say avoid the "classic cipher" stuff and go straight for the meat and potatoes. I can count on one hand the number of times I've had to break a simple custom transposition cipher or substitution cipher in a real product. They're just not used in reality, and if the client is using such awful "crypto" then that's already a major issue to flag to them - breaking it becomes a moot point.

You want to be looking into:

• Simple xor ciphers (one-time pad, two-time pad, repeated key xor)
• Stream ciphers
• Use of initialisation vectors (IVs) to avoid key re-use issues
• Stream cipher malleability
• Block ciphers
• Block cipher padding
• Block cipher modes of operation (Cipher Block Chaining especially)
• Issues with Electronic Code Book (ECB) mode
• IVs in block ciphers
• Malleability in CBC
• Padding oracle attacks
• Compression oracle attacks (e.g. CRIME)
• Hash functions (MD5, SHA1, etc.)
• Hash collisions and the birthday paradox
• Message Authentication Codes (e.g. HMAC construction)
• Asymmetric cryptography (e.g. RSA)
• Key exchange mechanisms (e.g. Diffie-Hellman)
• Hybrid cryptosystems (i.e. using asymmetric to send key, symmetric to encrypt)
• The list of common issues around SSL/TLS and PKI:
  • Not validating CA properly
  • Not validating common name properly
  • Not validating expiry or valid from dates
  • Allowing weak ciphers (e.g. NULL, DES)
  • Allowing weak certificate signatures (e.g. MD5)
  • etc... (see OWASP for more)

The list is pretty big, but it's stuff you can pick up from Wikipedia and other various places online at your own pace. If you understand half of the topics above at a level strong enough to identify a weak implementation, then you're very much on track. Almost every single product I've ever done work that has used crypto has had an issue in that list.