21

Assuming that I have binary file with code for unknow CPU can I somehow detect cpu architecture? I know that it depends mostly on compiler but I think that for most of CPU architectures it should be alot of CALL/RETN/JMP/PUSH/POP opcodes (statistically more than others). Or maybe should I search for some patterns in code specific for CPU (instead of opcodes occurrence)?

  • 1
    If you have a binary file but don't know for which CPU, how can you see opcodes? If you know how to translate from binary to opcode, then you already know which CPU you have. (Or at least which family -- e.g. Z80, Intel, ARM, Motorola MC-680XX.) – usr2564301 Oct 8 '13 at 12:48
  • Read the magic, then the file format. – Stolas Oct 8 '13 at 13:03
  • 1
    1) (Stolas) In embed often you don't have a magic or the magic is something they invented. 2) (Jongware) You can see opcodes (common patterns of bytes) without actually knowing what are them pretty much the same way you can determine if a file is compressed or encrypted without being able to decrypt or decompress it. – joxeankoret Oct 8 '13 at 13:14
  • @jongware I think that you confuse opcode with assembler instruction. – n3vermind Oct 8 '13 at 14:11
  • @n3vermind: .. if you don't know the CPU, then how can you be sure you are looking at 'opcodes'? ARMs, for example, would be easy (all opcodes are 4 bytes and most start with 0xE0), except you have Thumb modes to consider. A statistic approach may work -- but you always have the code/data dichotomy that makes disassembling hard even when you know the CPU type. – usr2564301 Oct 8 '13 at 15:04
16

When you have a hammer, all the problems look like nails...

I´ve studied something called Normalized Compression Distance - NCD - some time ago, and I'd give it a try if I had a problem similar to yours.

  1. I´d make a database of examples. Would take 20 programs for each architecture you want to know, with variable sizes, and save them.

  2. When confronted with a program that I wanted to know which architecture it is, I´d compute it´s NCD against all my examples.

  3. I´d pick the best (smaller) NCD and would then verify it if is was a real match (let´s say, trying to run it on the discovered architecture).

Update

I´ve always done in by hand, when it comes to NCD. How I did it:

  • you have 20 files for SPARC and you call them A01, A02, A03, and so on. Your x86 files: B01, B02, etc.

  • You get the unknown file and call it XX.

  • Choose your preferred compression tool (I used Gzip, but see remarks at the end of this answer).

  • Calculate NCD for the first pair:

NCD(XX,A01) = ( Z(XX+A01) - min(Z(XX), Z(A01) ) / max(Z(XX), Z(A01))

Z( something ) -> means that you compress the something with Gzip and get the file size after compression. For example, 8763 bytes, so Z(something) = 8763.

XX + A01 -> means that you concatenate things. You append the A01 file to the end of the XX file. In linux, you could do a ´cat XX A01 > XXA01´.

min() and max() -> you calculate the compressed size of XX and A01, and use the minimum and maximum that you get.

So you´ll have a NCD value: it´ll lie between 0 and 1, and use as many decimals places as you can, because sometimes the difference is in the 7th or 8th digit. It´ll be like comparing 0.999999887 to 0.999999524.

You´ll do that for every file, so you´ll have 20 NCD results for SPARC, 20 for x86...

Get the smaller NCD of all. Let´s say that the B07 file gave you the smaller NCD. So, probably, the unknow file is a x86.

Tips:

  • your unknow and your test files must have a similar size. When you compare a file with bigger or smaller ones, NCD won´t do it´s magic. So, if you´ll be testing files of 5 to 10k, I´d get test files of 2.5k, 5k, 7.5k, 10k, 12.5k ...

  • In my Master degree I got better results always using the smaller NCD value. The second best method was to do some voting: get the 5 smaller NCD results, and see which architecture got more votes. Ex.: smaller NCD were A03, A05, B02, B06, B07 -> B go 3 votes, so I´d say it´s a x86...

  • compressors based on the Zip construction have a limitation of 32kB: the way they compress things, they just consider 32kB at time. If your XX + A01 is bigger than this, Gzip, Zip, etc., won´t give you good results. So, for files that are bigger than 15 or 16kB, I´d choose another compressor: PPMD, Bzip...

  • 1
    Excellent idea. I've had good results for other classification problems in the past. – 0xC0000022L Oct 8 '13 at 18:12
  • 2
    @woliveirajr do you have any suggestion about tool or library for computing NCD? So far I have found CompLearn utilities which looks quite promising. – n3vermind Oct 8 '13 at 19:43
  • @n3vermind I´ve updated my answer: I think you could use CompLearn, but since I wanted more control (like which compressor to use), I´ve done a small program to suit my need. I explained how it works... – woliveirajr Oct 9 '13 at 11:58
  • 1
    @woliveirajr Do you have a link to your master thesis? I'd love to go over it – koukouviou Feb 22 '17 at 6:56
  • 1
    @koukouviou sorry, couldn't find it now (and it would be in portuguese, anyway). But here is one article that we wrote about it: inf.ufpr.br/lesoliveira/download/FSI2013.pdf -- Please let me know if I can help you or provide more information. – woliveirajr Feb 24 '17 at 17:08
12

There are some tools that can scan binary files for common opcodes found in various architectures. Binwalk's -A option does this for example (it scans for ARM/MIPS/x86 and several other architectures).

8

Typically, I try the most common CPUs first (ARM, PPC, MIPS and AVR), try to find if any of the plain strings says something about the processor, etc... And, when all else fail, I give a try to what you're asking for: statistical analysis of opcodes (if I'm sure it isn't neither encrypted nor compressed).

I recommend you to read the Alexander Chernov and Katerina Troshina presentation "Reverse engineering of binary programs for custom virtual machines". Writing a tool like the one they wrote must be very hard (I guess) but writing a tool to try to determine which CPU seems to be compiled for using the techniques described in that presentation is not that hard (as long as you can collect enough samples for multiple different architectures).

7

My lazy hack: a small Python script which calculates bigram and trigram counts. I then search for a couple of the most common sequences on Google (quoted hex). Quite often I manage to find some hex dumps and can figure the CPU from the context. It would work even better if Google could search by raw binary values...

  • 1
    May be I'm late to the party, but this site has Python API and surely can search raw binary values: binar.ly/search – Anton Kochkov Jul 20 '16 at 20:36
  • @AntonKochkov thanks, looks intersing! too bad it seems to index only malware... – Igor Skochinsky Jul 21 '16 at 7:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.