How do I determine code coverage when fuzzing

Question

Let's say I'd like to begin fuzzing Acme Corp's PDF Reader. I'd like to try to follow what Miller did by downloading a bunch of benign PDFs and mutate them.

Miller began by reducing his corpus of PDF samples to a minimum by pruning samples that had similar code coverage.

How is that specific step done? That is, how did he determine what was a similar code coverage?

I can imagine a tool that traces execution and records JMP/CALLs to get an execution graph, and I suppose you could diff those graphs. But what about JIT code? Wouldn't those graphs be very different since the JIT would likely be in different locations in memory?

Answer 1

Not sure how it fares against application with JIT compiled code, but peach has a minset utility to make a minimal set of files with highest code coverage:

This tool will run each sample file through a target program and determine code coverage. It will then find the least number of files needed to cover the most code. This will be the minimum set of files that should be used when fuzzing.

But as far as I can see it uses the method you proposed, monitoring hits of all basic blocks of the application. It uses a pintool to do this.

Answer 2

Although it is heavily coupled with GCC, the gcov project is popular as a Linux code coverage tool. It however requires compiling your program with the -fprofile-arcs -ftest-coverage flags, which may not be an option if targeting closed source software. More information may be found here: http://gcc.gnu.org/onlinedocs/gcc/Gcov.html