Skip to main content
2 of 4
added 1403 characters in body
Peter Andersson
  • 5.7k
  • 1
  • 32
  • 49

Compiler

The choice of a compiler has minimal effects on the difficulty to reverse engineer your code. The important things to minimize are all related to information leaks from your code. You want to at least disable any runtime type information (RTTI). The leakage of type information and the simplicity of the instruction set of the virtual machine is one of the reasons CLR and JVM code is easier to reverse engineer. They also have an JIT which applies optimizations to code which may reduce the strength of obfuscation. Obfuscation is basically the opposite of optimization and a lot of obfuscations are solved by first applying an optimization pass.

Debugging information

I would advice you to also turn off any debugging information, even if it doesn't leak any majorly important information today it might do so tomorrow. The amount of information leakage from the debug information varies from compiler to compiler and from binary format to binary format. For instance Microsoft Visual C++ keeps all important debugging information in an external database, usually in the form of a PDB. The most you might leak is the path you used when building your software.

Strings

When it comes to strings you should definitely encrypt them if you need them at all. I would aim to replace all the ones that are for error tracing and error logging with numeric enumerations. Strings which reveal any sort of information about what is going on right now in your binary needs to be unavailable. If you encrypt the strings, they will be decrypted. Try to avoid them as much as possible.

System APIs

Another strong source of information leakage is imports of system APIs. You want to make sure that any imported function which has a known signature is well-hidden and can not be found using automatic analysis. So an array of function pointers from something like LoadLibrary/GetProcAddress is out of the question. All calls to imported functions need to go through a one way function and needs to be embedded within an obfuscated block.

Standard runtime libraries

Something a lot of people forget to take into consideration is the information leaked by standard libraries, such as the runtime of your C++ compiler. I would avoid the use of it completely. This is because most experienced reverse engineers will have signatures prepared for a lot of standard libraries.

Obfuscation

You should also cover any critical code with some sort of heavy obfuscation. Some of the heavier and cheaper obfuscations right now are CodeVirtualizer/Themida and VMProtect. Be aware that these packages have an abundance of defects though. They will sometimes transform your code to something which will not be the equivalent of the original which can lead to instability. They also slow down the obfuscated code significantly. A factor of 10000 times slower is not uncommon. There's also the issue of triggering more false positives with anti-virus software. I would advice you to sign your software using a reputable certificate authority.

Separation of functional blocks

The separation of code into functions is another thing which makes it easier to reverse engineer a program. This applies especially when the functions are obfuscated because it creates boundaries around which the reverse engineer can reason about your software. This way the reverse engineer can solve your program in a divide and conquer manner. Ideally you would want your software in one effective block with obfuscation applied uniformly to the entire block as one. So reduce the number of blocks, use inlining very generously and wrap them in a good obfuscation algorithm. The compiler can easily do some heavy optimizations and stack ordering which will make the block harder to reverse engineer.

Finally

Study your software in something like the free version of IDA. Your goal is to make sure that it becomes virtually impossible for the reverse engineer to find a steady mental footing. The less information you leak and the more changing the environment is, the harder it will be to study. If you're not an experienced reverse engineer, designing something hard to reverse engineer is almost impossible.

If you're designing a copy protection system prepare for it to be broken mentally. Make sure you have a plan for how you will deal with the break and how to make sure the next version of your software adds enough value to drive upgrades. Build your system on a solid ground which can not be broken, do not resort to generating your own license keys using some custom algorithm hidden in the manner I described above. The system needs to be built on a sound cryptographic foundation for unforgeability of messages.

Peter Andersson
  • 5.7k
  • 1
  • 32
  • 49