How are vulnerabilities (especially buffer overruns) found in the wild?

Question

I hope this question is not OT for RE, but I'm rather curious as to how vulnerabilities are usually found.

Of course I'm aware that companies are doing code audits to identify security problems but I doubt that the results of such audits are publicly made available. Another way to find potential attacks is of course thinking about the details of a particular technique and finding its weakness (a timing attack would probably qualify as such).

However, in the case of buffer overruns, I'm always wondering how people find out about it. I mean, if there is a release of some software, reversing it and hoping to find a buffer overflow this way seems rather hopeless to me, considering how much work this is. If your software crashes because of some special input, then of course this can be analyzed and might result in a security vulnerability. So are malware author just monitoring various sources (bug reporting site or similar) in the hope of hearing about such cases to look into it? Somehow I can't believe it.

Answer 1

Most vulnerabilities in closed-source products are found via fuzzing and static reverse engineering.

reversing it and hoping to find a buffer overflow this way seems rather hopeless to me, considering how much work this is

Typically you don't need to analyze the entire program, but only the entrypoints for user-input. Yes, it's still a lot of work, but if the results are fruitful then that's all that matters.

So are malware author just monitoring various sources (bug reporting site or similar) in the hope of hearing about such cases to look into it?

Most malware does not propagate via security exploits, but by simple social-engineering ("here, run this .exe"). For malware that does exploit security vulnerabilities, the authors either rely on modules from Metasploit or buy exploits from the black market. It's extremely rare for a single person to be both a vulnerability finder and a malware author.

Answer 2

Fuzzing finds by far the majority of buffer overflows. Reverse engineering can both help the fuzzing and find additional issues on its own. If the fuzzer knows about 'special' values that will cause the code to branch differently or exit parsing earlier without their presence, it can weight the inclusion of those values, guarantee that the input will get past a certain point, or fill in a CRC to always be correct.

On it's own, reverse engineering can be very good if you start with an idea of what kind of defect you want to find. For example, if you start with the idea that people who use atoi() will not expect negative numbers, it is easy to find those places in the code and flow forward to see what happens with the data. Another example might be to start with the assumption that memory allocation size could be integer overflowed so, look at each call to the allocation and see if the computation preceding it has the necessary checks. If not, see if the input can influence the allocation to reach a defect.

Another way to use reverse engineering is to let someone else do the defect finding for you. Just compare what was patched when the vendor releases an update and you'll see what was repaired.

As Jason said, the people writing the malware are not typically the same people that are finding the defects or even developing the exploits.