IDA plugin/script for generating signature statistics (for all sigs) for a target?

Question

I have more than 40 different IDA signature files (besides the built-in ones) and sometimes I don't want to go through the hassle of examining what signatures may be relevant for a given target. Instead I just want to apply all signatures, one at a time, and have IDA tell me which ones had the most matches in the target. Doing this manually is very tedious because

1. IDA only allows applying 1 signature in each apply operation
2. The application order of signatures influences how many matches a given signature generates. For example, when sig A is applied it may mean sig B misses some matches that A has matched before it.

I imagine the plugin/script would do something like this: apply sig A to the target database, count matches, then remove the application of sig A and apply B, count matches, then remove the application and apply C and so on.

Has nobody else ever had this idea? I have been unable to find a plugin or script that does this, and I would rather not write my own if one already exists.

Answer 1

If you are talking about IDA FLIRT signatures, unfortunately sooner or later you will probably hit their limitations, and, or, get frustrated with the results if you want high accuracy.

First you can read about IDA's system in detail here: https://www.hex-rays.com/products/ida/tech/flirt/in_depth.shtml

As you can see it only matches the first 32 bytes of functions. And then it uses CRC16 so there is fair chance of collisions.

The biggest problem with FLIRT is that it's pretty ambiguous.
It's prone for false matches. And, or, if the first 32 bytes of a matched function haven't changed but then the rest has it will still match. This is often a good thing, but then the behavior is unpredictable. Also the system within IDA lacks anything more then a minimal UI. It would be nice for example if IDA tracked and showed you a list of both what matched and what doesn't match if you wanted it. Although supposedly in 6.4 they overhauled and updated the setup so it might have been improved.

Now it's not terrible; It's better then nothing and is pretty useful over all. But because of these problems, you'll probably want to make your own.

I made my own signature system that walks over each function of the source signature target and copies all the non-relative bytes (I.E. it skips the offset bytes from JMP, MOV, CALL instructions, etc.) and makes a 64bit hash of the whole thing. I only save unique user named functions and skip small ones (below a byte count cut-off point). There is no point in saving redundant signatures.

This way I know that later on if I have a match it more then likely a good and unique match. It's much more accurate and has little chance of false and redundant matches. Also it's done from a plug-in that is about as fast if not faster then IDA's own sig system. It also allows me to add UI lists of matches, non-matches, stats, etc., what ever I want.

Answer 2

You may be able to make use of snapshots to make this happen from a plugin. I don't think the database snapshot functions are exposed in IDC or IDAPython.

General algorithm on a freshly opened database would be something along the lines of

libs_before = iterate over all functions to determine initial count of library functions
snapshot_t ss;
::qstrncpy(ss.filename, "tempsnap", 9);
qstring errmsg;
take_database_snapshot(&ss, &errmsg);
//use platform directory iterator on IDA sigs directory
for i in list_of_sig_files
   int signum = plan_to_apply_idasgn(i);
   while (i--) {
      apply_idasgn(1);  //may need to play around with this
   }
   libs_after = iterate over all functions to determine count of library functions after sigs applied
   msg("%d signatures matched from %s\n", libs_after - libs_before, i);
   restore_database_snapshot(&ss), mycb, NULL);

where a minimal mycb is:

void idaapi mycb(const char *errmsg, void *ud) {}

obviously there is some code for you to fill in, but the idea to get an initial count of library functions in the binary and take a snapshot. Next iterate over the names of all sig files and apply each one in turn. Following the application of each signature file, generate a new count of the library functions in the database. The difference between the new count and the original count is the number of signature matches. Finally restore to the snapshot you took to roll back the changes made when applying the sigs.

You can count library functions by iterating over all functions (see get_next_func) and for each returned func_t *f, check (f->flags & FUNC_LIB) != 0.

Hopefully that puts you on the right track

Answer 3

There is a plugin called BinDiff

If you have renamed Sub_Routines in one IDA DB, you are able to run BinDiff between your current one and a new DB that hasn't been refactored yet. The only down side is it's a little time consuming for binaries that are 30mb+ in size.

http://i.imgur.com/dTPEyHq.png