What are the targets of professional reverse software engineering?

Question

At the professional level, for what purpose is reverse software engineering used? What software is targeted and why?

For reasonably complex compiled code that's doing something novel, making meaningful insights into how that code operates via reverse engineering seems like it would be enormously intensive of expertise, labor, and time. In particular, I imagine that hiring competent assembly programmers is extremely difficult and possibly expensive. And yet, I haven't the foggiest idea where entities with the resources to do so would want to spend those resources.

This is my list of possibilities...

1. Writing malware
2. Writing counter-malware
3. Maybe analyzing competitors products?

It's not a great list. What is the reality here? What sort of software justifies the expense to be reverse engineered?

See the comments on 0xC0000022L's answer for some refinement of the question.

Answer 1

Reverse engineering is commonly used in many ways. Here is a list of just some of the most common professional activities where reverse engineering is involved.

1. Malware research. Withouth doing reverse engineering, it's hard to determine what an actual piece of malware does, how and, more important, how to clean it or prevent infections.
2. Plagiarism detection: Reverse engineering is often legally used to determine if company or particular X plagiarized code from company or particular Y.
3. Forensics.
4. Compatibility. Reverse engineering supported, and still does, many open source projects like Samba or Star/Libre/OpenOffice. Without professional (and hobbistic) reverse engineers, nobody would have been able to write a software like the previous ones as there was no specification available at all for the protocols or file formats involved. If you're using Libre/OpenOffice or MacOSX or Linux, you're actually using the result of the work of many reverse engineers during many years.
5. Vulnerability research/development. In order to discover undocumented features, vulnerabilities or backdoors left by the company, programmers or a 3rd party, reverse engineering is naturally required.
6. Modification of legacy programs. This is one of the most other common tasks I have seen in this field: company X bought program Y but the company producing it, Z, disappeared and they have no source code at all. That company X contracts reverse engineering services to fix bugs and/or adapt the old (dead) application for their own purposes. If the program was also protected, the reverse engineers would also need to first break the the copy protection.
7. Analysing competitors products. Naturally, this is a very common thing. Specially when talking about hardware products. And it, by the way, happens in all fields. Do you think that Porsche doesn't reverse engineer Ferrari cars and the other way around?
8. Source code recovery. Company X lost the source code of one of their programs/products for whatever reason and they only have the binaries. That company contracts reverse engineering services to reconstruct/recover the source code.
9. Analysing foreign governments hardware and software. You get the idea.
10. Binary patches development. Some products still evolve but the providing companies are very slow fixing bugs (Oracle comes to mind...) or simply don't care. In this case, reverse engineering services can be contracted to develop binary patches to be applied to the product in order to fix bugs or vulnerabilities (patches to be made again on each release).

I could continue writing down more professional task that reverse engineers often do, but I think this list gives you a general idea.

Answer 2

4. Finding undocumented features of a product.

   • for instance for locating a registry key or setting to turn on a debug mode.

5. Finding how exactly a poorly documented feature works.

6. Debugging

   • sometimes i find myself loading my own software in IDA to find what code the compiler produced, either to find bugs, or to see how an optimization worked out.

7. Cracking copy protection

   • keygens are usually created by reverse engineering an application's licensekey validation algorithm.
   • sometimes a company may have lost licene codes to some ancient piece of software.

8. Evaluating the security of a closed source product.

9. Modify an existing product

   • like flashing your own custom rom to a phone.

10. Interface current products with ancient software.

Answer 3

As a professional Reverse Engineer I have reveresed for:

• Troubleshooting
• Exploit Development
• Malware Analysis
• Implementing badly documented API calls.
• Plagiarism , they knew of an undocumented Kernel call within the Nokia phonebase. Thus we reverse engineerd this and created a program that used this call as well.

Overall reversing increases your knowledge in programming.

Answer 4

A key purpose in reverse engineering it to recover an abstract design of the artifact.

A source program is an implementation of a concept. It contains a huge amount of code/gates that achieve an effect, and then optimizations on top of that to achieve the effect reasonably well (space, time, heat, area, power, ...). It is focused on good execution, not good documentation of the concepts.

That's all well and fine for the running product. But if you want understand the code you have (and you have made the mistake of throwing the design away [everybody has, don't get me started on the stupidity of this]), then abstracting away all that detailed implementation, leaving behind the abstract structure and computation, is extremely helpful.

Here's technical paper on how to do this in a regular way, that I wrote on the topic (last century [!], still good IMHO): Reverse Engineering is Reverse Forward Engineering. This paper takes low level assembly code from a real time OS, and abstracts it back into the synchronization primitive it implements. In essence what it says is, "This concept was realized in the code like this at these places in the code".

This idea is partially derived from Rich and Water's famous(?) mid 1980's research on The Programmer's Apprentice. In their case they were focused on finding realizations of abstractions in code, by doing what amounts to data flow graph matching. (There have been a variety of other papers and Ph.D. theses suggesting similar ideas). My RE paper suggests grafting an entire design model on top of low level recognition.

My company recently has recently implemented Rich&Water's vision, for Dow Chemical, extracting process control models from low level industrial controller code. Dow is using this to migrate legacy code that run chemical plants, to modern controllers. See SD building tools for Dow Chemical. This only announces the work, but we've been at it for a few years and are getting good results. We hope to be writing this up for technical journals soon, as well as applying this to more conventional languages such as C.

I assert that all these ideas apply to hardware and codesign too.

Answer 5

13. Writing cheats for games.
14. Writing an anti-cheat for games.
15. Looking for badly written code to exploit or to fix and patch it.

Answer 6

10. Honing your skills and keeping up-to-date with them (as well as with using the tools).
11. Curiosity. Yep, doesn't have to be purely goal-driven like 4. and 5. in Willem's answer.
12. Establishing compatibility with a legacy product, file format or architecture. Motivation can be 11 in that case or 4 or 5 or possibly something completely different.

Answer 7

I have carried out reverse engineering projects professionally a fair bit. Generally we are commissioned to:

1) Determine how hard it would be to create a crack for some piece of software (by the developers of that software).

2) Understand a specific exploit (in the security sense, e.g. a buffer overflow). In other words, "here's the email we got; tell us what happened", or "here's the exploit. How, exactly, does it work and how can we defend against this kind of exploit in future?"

3) Find bugs in a piece of software by examining the binary without reference to the source code, as part of a larger security audit of the code. This is surprisingly useful because some kinds of dangerous behaviour that aren't obvious in code can "leap out" at you in the binary, e.g. complex preprocessor macros causing dangerous behaviour in C/C++. So, counterintuitively, we sometimes use reversing because it's faster. Check out IDA pro and Hopper for the sorts of "quick reversing" features that help in this area - decompiling to pseudocode, finding all calls to X or just searching the string table then finding all references to a dangerous/suspect looking string, and so on. Sometimes just running 'strings' on a binary can give you ideas as to where the bugs might be; following that up with a real, pro reversing tool can help. This isn't restricted to C/C++/Objective C - if we're working in .Net or Java, we'd use .Net Reflector or the Java Decompiler and other environments have their equivalents.

Sometimes we use reverse engineering to help us in a larger project. For instance, we might want to test a component in isolation; to do this without the source code we might want to edit the binary to introduce logging calls or shortcut some behaviour.

Then there's fuzzing, of course, but that's probably not really reversing in the true sense, although it can involve examining code at the assembly level, to understand why the code crashed / where the bug is / whether it's a security problem.

There are many, many reasons why reverse engineering is helpful even in a general development context. In the security field, it's pretty much essential.

In terms of professional development, if you're in the security field and you have good reversing skills, you will probably be at the upper end of the pay scale. You need to be able to do the other stuff, but if you have reversing on your CV - and you can back it up - then getting hired shouldn't be a problem (in the security field at least).

Answer 8

There are already a lot of great answers. If I may add my two cents. Reverse engineering software is akin to the mechanic or tinkerer whom just enjoys taking things apart, understanding how they work, putting them back together, and possibly modifying their subject to adapt its behavior so it is more to their liking. There is no shortage of examples in the physical realm.

Any Electrical Engineer, hardware designer, and the most eliete QA engineers are talented reverse engineers; which is to say, they have an expertise in debugging/analyzing software/hardware implementations. I interned at AMD as I worked my way through school and my debugging skills not only helped me advance to more serious roles within the company, but it is also really fun! Like a game or puzzle that you can play for as long as you want. The challenges will never stop coming.

My favorite example of a legitimate need for reverse engineers is NASA. They are not the only outfit where the phrase "mission critical" applies, where people's lives depend on the quality of their work, but it is the only one that has to fix problems even if their product is hurtling through space. They have to set and freeze product versions years before that software / hardware ever goes to the launch pad. That may seem silly, because the phones in our pocket have more computing power than the entire shuttle, but they can't use modern technology. It's too risky. They have to accept what is available at the time that the specs are frozen and that's it - lest they incur the scheduling and workload that upgrading, testing, fixing, re-testing, verifying again, simulating realistic conditions to determine failure points, fixing it, and starting over.... it is serious business and those engineers do not take their responsibility lightly.

Would you be surprised to learn that NASA projects have used Microsoft OSes on hardware went into shuttle components? It is not much different than using a M$ OS as the foundation for, say for instance, a point-of-sale terminal or for the control system at a power plant. Now consider a scenario where the OS, the hardware, or some portion of third-party software is found to have a flaw in it. Moreover, imagine that the vendor does not care about the flaw to the degree that NASA does and decides that there is no fiscal motivation for them to fix the issue - even if lives depend upon that component not failing. NASA is stuck.

At this point their options are to either replace the component and start their processes all over - a huge cost in time, man-hours, and no guarentee that they won't discover another critical flaw after upgrading; or, instead of upgrading, they can reverse engineer the hardware/software failure and determine if they could fix the problem themselves - maybe it would take some curcuit re-design, changing the values of a few resistors so that voltage levels on the memory bus stayed within required limits under the extreme conditions that were making them fail or maybe there is a binary patch that they can apply that to mitigate the error state they discovered where life-support systems began failing or a probe's batteries drained and left it unable to re-charge itself.

I'm not totally making these scenarios up. They come from stories I have been told, by friends who worked at JPL, my engineering professors when I was in the University, or colleagues that have worked with in the tech industry. All this, and more, has been surmounted by NASA's engineers. It did cost quite a bit more than India paid (tip of the hat to them too), to do what they have done, but the first time through is always more costly.

Technically, NASA engineers would be violating one or more of the intellectual property laws that vendors claim they have (they may or may not actually have those rights; I"m not trying to debate that here). If you bought a corvette, you would be completely within your rights to bore out the cylinders to change your car's performance profile. Different tires. Modified suspension. Tweaked timing and mixture ratios in the fuel injection system. The famous Shelby Cobra is not only modified, but also remarketed with its aftermarket modifications- all entirely legal.

If you bought a radio and wanted to make some modifications, so that it was water proof and you could take it on the lake with you. Amplify it's output so that it would still be audible over the boat engine, or anything of that sort... legal.

Now, jailbreak your iPhone, modify your xBox or the Kinect system that came with it, or, in NASA's case, modify the OS and/or re-engineer the memory bus on a motherboard, to ensure the safety of a human mind you. All of these activities have evoked fierce reactions from the vendors that, for whatever their reasoning, do not want those activities to be continue nor become more widespread. Ultimately, this led to the DMCA laws that we have today being passed. Before that, even if you were clicking "I agree" to the rediculus terms and services that were put before you there was still the question if that contract was legal enforceable.

The DMCA, which does nothing to disuade the massive blackmarkets in China or South America, carries harsh penalties and has been treated as a kind of "hunting license" for those that wish to go after consumers can and have. Should it be illegal to make unlicensed copies of someone elses music; to sell, trade, or even give it away?

Clearly, that should be illegal. Artwork, books, patented processes, protected trademarks; yes, those should be protected too. Software and hardware; again, yes the the author of software should be able to enforce resonalbe aspects of their licensing that protect the the investment that have made to create said software or hardware. Explicitly, it is, and should remain, illegal to puchase one copy legitimately, turn around and make copies, and then sell/trade/or give the unlicensed copies. Microsoft, VMWare, Adobe, and similar big companies should retain that right.

Now, once I have licensed the software or hardware can I make modifications and then sell/trade/or give away the software/hardware that featured my modifications. That depends. It i, and should be illegal, to profit from derivatives works beyond the limit to which you originally licended the art (i.e. software). For example, you could not draw the likeness of a Disney character, change the name, and then claim it as your own and profit from it's use. I believe that Mikey no longer protected, but when he was that was and should be illegal.

Should a computer hobbyist, like the ones that we credit for founding the industry today, be allowed to to explore, tinker, change, discuss, show, or teach any manner of activity that they desire on their legally licensed software and hardware? I won't answer that, but I will say that the DMCA says "no". In fact, the same tools that a reverse engineer would use for debugging the operating system are the very same tools that the prosecution will cite as evidence of illegal activity.

Let's be a little more giving for someone prosecuting a DMCA case where the accused was know to have accessed the cryptographic keys that, among other things, would allow the defendant to make copies of protected art (e.g. music and movies). There are some legitimate reasons that could explain the defendants actions, but not many and the illegal activities that could be pursued are costly to those that are trying to protect themselves. At the end of the day, even though they tried their best, there is no way to actually prevent anyone from accessing those cryptographic keys.

It reminds me of the tag on a mattress waring of insanely harsh penalties for removing said tag, but there is no protection for the tag. Just threats and overreaching penalties that are used to make examples out of as they stoke the fire and declare a witch hunt.

Should Apple be able to prosicute a security research whom respectfully, and sincerely trying to help and support Apple make better products, be prosicuted by the technology giant? Hell no, but you better believe that they tried - that happened. The employees of Target or Home Depot that warned of potential dangers; where are they now? They were fired, run-off, considered a nuisance to the company, shunned, and labeled as "not a team player". In the wake of the data breaches that Target, Home Depot, and countless others (certainly at least one of these events has impacted your life) it is more apparent now than ever that reverse engineering is not only a right, but an aspect of our emerging social structure that is going to be needed more than ever to 1) protect the innocent consumers and digital citizens of the world like yourself and 2) hold accountable those whom are responsible to take reasonable measure to protect the community that entrusted them with their well being (granted in exchange for a service).

The NSAs, the CIAs, the DeutcheBanks, the HealthCare.govs, and the hospitals that we all wanted and empowered to do what they have done. It is the reverse engineer who will become the modern freedom fighter, tomorrows activist, that is going to be on your side, the little guy, when power corrupts and greed lets way to disregard for the responsibilities that were implicit in the power granted to those organizations. To be just a bit more dramatic about it; reverse engineers are the priests of a new religion. We should honor them, applaud their bravery standing up to those drunk on power, and protect them when they risk the privacy of their family to step into the line of fire, on our behalf, because it is "the right thing to do".

If you know one of these individuals- buy them a beer tonight and say thank you. As much as any soldier, they are going to be the ones that protect us when the next digital tragedy is impending.