5

In the pursuit and development of malware detection algorithms, often a big sample set of both malicious and benign samples is required. Both machine learning or similar automated techniques, as well as manual or partially manual signature generation, often require a good and varied example set of benign samples that are commonly mistaken as malicious.

Those samples are usually being automatically analyzed and then provided to a Reverse Engineer for further scrutiny, analysis and improvement of said malware detection algorithm.

Although finding malicious samples is frequently discussed (see multiple questions), discussion about benign sample sources seems lacking.

What are good benign sample repositories/feeds, preferably focused on potential/frequent false positive samples? Other sources or "retrival methods" (scraping) are also welcome!

5

You can find some of the samples from our work on deobfuscation at hxxp://www.cs.arizona.edu/projects/lynx-project/Samples/ (s/x/t/). The programs---a mix of Linux and Windows binaries---are all benign (you'll have to take my word for this :-), but use a variety of techniques commonly used by malware. These samples formed the basis for the results reported in the following papers:

  • Yadegari et al, "A Generic Approach to Automatic Deobfuscation of Executable Code", Proc. IEEE Symposium on Security and Privacy, May 2015. These consist of programs obfuscated using tools such as VMProtect, ExeCryptor, and Themida; and several that use return-oriented programming.
  • Qiu et al, "Identifying and Understanding Self-Checksumming Defenses in Software", Proc. Fifth ACM Conference on Data and Application Security and Privacy (CODASPY), March 2015. This is a collection of programs that use a variety of techniques, including self-modifying code, to implement self-checksumming-based anti-tampering.

We used to have a link to these samples on the project web page, but Google decided that we were distributing malware and put a big red warning next to their link (talk about false positives!), so we ended up removing the link. This is also the reason the link above has been disguised with "hxxp".

  • Thanks a lot! Would you be kind enough to explain the origin of the samples? Rough quantities would also be appriciated! – NirIzr Sep 6 '16 at 18:33
  • The obfuscated code consists of several small programs we wrote ourselves (binary-search, bubble-sort, matrix-multiply, etc.); and some quasi-malware -- code whose source we got from VX Heavens and which we then obfuscated using Themida etc (I call them "quasi-malware" because they don't have the actual malicious payload). The ROP code is a collection of a few small programs we wrote ourselves (bubble-sort, factorial, fibonacci, matrix-multiply). The self-checksumming programs are all based on an open-source MD5 program. Please see the publications mentioned for details. – debray Sep 6 '16 at 18:56
  • Awesome, thanks! I'll upvote your answer in a few hours, but I'll wait some more before I accept any answer – NirIzr Sep 6 '16 at 18:59
  • @NirIzr: my student mentioned that he thought that some of the samples from VX Heavens might still have questionable content -- he thought he had some problems with some of them. Just to be safe, I've removed all of those samples from that website, so what is left are just programs we wrote ourselves and are certain about. Please discard what you took previously and grab them again. Sorry about that. – debray Sep 8 '16 at 18:40
5

Malicious samples

My favorite source for malicious samples is a database i assembled myself by regulary pulling the free examples provided by VirusSign. Please not these samples do require registration, but come with reports indicating the type.

There may be no better archive for malicious samples than VirusTotal. If you don't happen to gain (intelligence) access there, you can still try open alternatives like Open Malware.

Generally, a good list for places to find malicious samples is here: LENNY ZELTSER

Benign Samples

As for benign samples, I propose extracting benign executables from a fresh OS installations. You can simply install the target in a virtual machine and get a script to extract them.

Another source is crawling sites like PortableFreeware. Please make sure you check if executables collected this way to work in your environment.

Possible false positives

Most malware archives label their samples based on the detection ratio and labels provided by different ant-malware products (like VirusTotal). Asssuming samples with very low detection ratio and generic labels to be false positives is probably the best way to go here. Also, data collected from HoneyPots may yield relatively higher false positive rates.

I often used the executables included in the Sysinternals Suite as a kind of 'false positives'. They are benign binaries utilizing system-near features in order to expose system internal data.

On a sitenode, please take time evaluating your dataset and finding biases hidden within. A lot of current research suffers from biased sample sets and unreproducible results.

  • Thanks! For the record I was asking only for white samples – NirIzr Sep 5 '16 at 7:24
0

Some of the techniques i've also came up with, for the sake of completeness (I won't accept my own answer):

download from github

Some Github repositories have multiple executables either as needed utilities or as build output. Scanning Github for those using a GithubDownloader proved being slow but effective, and I results are really likely to be benign.

collect all executables/hashes from accessible machines

Since I'm doing this as part of my job, I'm able to ask the IT department to collect hashes and samples from the multiple machines we have at my office. This was a good way to collect many executables for multiple versions of OSes with ease. Collecting hashes and downloading the ones available from VT was also a possibility (perhaps after further scrutiny).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.