Firmware analysis and file system extraction?

Question

I'm trying to analyse the firmware image of a NAS device.

I used various tools to help the analysis (binwalk, deezee, signsrch, firmware-mod-kit which uses binwalk AFAIK), but all of them have been unsuccessful so far.

For example binwalk seems to generate false positive regarding gzip compressed data and Cisco IOS experimental microcode.

Scan Time:     2013-08-27 14:52:15
Signatures:    196
Target File:   firmware.img
MD5 Checksum:  4d34d45db310bf599b62370f92d0a425

DECIMAL         HEX             DESCRIPTION
-------------------------------------------------------------------------------------------------------------------
80558935        0x4CD3B57       gzip compressed data, ASCII, has CRC, last modified: Fri Oct  4 17:37:33 2019
82433954        0x4E9D7A2       Cisco IOS experimental microcode
145038048       0x8A51AE0       gzip compressed data, ASCII, extra field, last modified: Mon May 26 20:11:40 2014  

When trying to decompress the data I got the following error using gunzip/gzip

gzip: 4CD3B57.gz is a multi-part gzip file -- not supported

According to gzip FAQ (http://www.gzip.org/#faq2) this is due to a transfer not made in binary mode which has corrupted the gzip header.

It looks more like a false positive from binwalk to me mostly because the magic number used to identify gzip data can easily trigger false positive and the dates are wrong.

I also ran strings and hexdump command in order to have an idea of the contents of the file and try to identify known pattern but it didn't help much so far (I probably lack experience in that type of thing here).

The only non-gibberish/identifiable strings are located at the end of the firmware image.

00000000  f5 7b 47 03 d5 08 bf 64  ba e9 99 d8 48 cf 81 18  |.{G....d....H...|
00000010  b1 69 1e 2c c2 f3 46 6b  53 2b b7 63 e8 ce 78 c9  |.i.,..FkS+.c..x.|
00000020  87 fd b8 68 41 4d b2 61  71 cb cc 75 eb 8c e0 75  |...hAM.aq..u...u|
00000030  25 d1 ec bd 6d 46 e8 16  37 c6 f5 2e 2a e0 dc 07  |%...mF..7...*...|
00000040  65 b1 ce 7f 20 57 7c d7  cb 1d 91 fc 05 25 ad af  |e... W|......%..|
00000050  58 56 ff 13 4d 03 95 7f  ad 58 0e 84 85 2f 73 5c  |XV..M....X.../s\|
00000060  d9 19 d4 d4 2c 27 be c6  45 f2 9f a4 b1 e1 04 f1  |....,'..E.......|
00000070  c1 28 17 9c e1 f7 9d 2b  63 c3 7d e1 95 56 06 05  |.(.....+c.}..V..|
[...]
09ec9d60  4b 29 75 20 46 6e fb e3  0f 14 d4 93 54 8e 4f bb  |K)u Fn......T.O.|
09ec9d70  4b ab 91 bf e7 8a b9 4e  c8 ff 87 17 93 19 e9 3f  |K......N.......?|
09ec9d80  70 fe a6 9f d3 36 48 83  34 48 83 34 48 83 34 48  |p....6H.4H.4H.4H|
09ec9d90  83 34 48 83 34 48 83 34  48 83 34 48 83 34 48 83  |.4H.4H.4H.4H.4H.|
09ec9da0  34 48 83 34 48 83 34 48  83 34 48 83 34 48 83 34  |4H.4H.4H.4H.4H.4|
09ec9db0  48 83 34 48 83 34 48 83  34 48 83 34 48 83 24 a7  |H.4H.4H.4H.4H.$.|
09ec9dc0  ff 07 e9 0d 37 73 00 20  08 0a 69 63 70 6e 61 73  |....7s. ..icpnas|
09ec9dd0  00 00 10 00 54 53 2d 35  36 39 00 00 00 00 00 00  |....TS-569......|
09ec9de0  00 00 00 00 33 2e 38 2e  33 00 00 00 00 00 00 00  |....3.8.3.......|
09ec9df0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
09ec9e14

It is the first time I'm going through that type of exercise and I'm not sure what I should do next. The image seems to be obfuscated somehow (that might be a wrong assumption).

Do you have suggestions/tricks that could help me make some progress?

Answer 1

I've been dissecting the firmware for another type of embedded device for a while and thought I'd see if I could find anything out. After a few hours I figured it out! There is a hard way and an easy way that I found only after digging the hard way. This is a long post, but I hope it will help others in similar ventures.

A little Googling and I found http://wiki.qnap.com/wiki/Firmware_Recovery describing full firmware recovery methods and another page on the same wiki for "Manually Updating Firmware" with command line examples. A couple of things stuck out ...

• The NAS OS has a script for processing a firmware update image:

  # /etc/init.d/update.sh /mnt/HDA_ROOT/update/TS-209_2.1.2_build1031.img
  

• There is a checksum embedded in the binary and THEN there is this line in the output:

  "Using 120-bit encryption - (QNAPNASVERSION4)"
  

I went down 2 paths: the hard way, and the very easy way ...

The Hard Way (but with useful tips)

I downloaded the TS-569 full system recovery image from the Firmware Recovery page which took almost 2 hours for 500MB. Now I had to figure out what I was working with:

# file F_TS-569_20120628-1.2.2.img
F_TS-569_20120628-1.2.2.img: x86 boot sector; GRand Unified Bootloader, ...

A full disk image which looks like this:

$ fdisk -l F_TS-569_20120628-1.2.2.img
                      Device Boot      Start         End      Blocks   Id  System
F_TS-569_20120628-1.2.2.img1              32        4351        2160   83  Linux
F_TS-569_20120628-1.2.2.img2   *        4352      488959      242304   83  Linux
F_TS-569_20120628-1.2.2.img3          488960      973567      242304   83  Linux
F_TS-569_20120628-1.2.2.img4          973568     1007615       17024    5  Extended
F_TS-569_20120628-1.2.2.img5          973600      990207        8304   83  Linux
F_TS-569_20120628-1.2.2.img6          990240     1007615        8688   83  Linux

Separate out the partitions (or you could write the image to a spare disk):

# dd if=F_TS-569_20120628-1.2.2.img bs=512 of=part1 skip=32 count=2160w
# dd if=F_TS-569_20120628-1.2.2.img bs=512 of=part2 skip=4352 count=242304w
# dd if=F_TS-569_20120628-1.2.2.img bs=512 of=part3 skip=488960 count=242304w
# dd if=F_TS-569_20120628-1.2.2.img bs=512 of=part5 skip=973600 count=8304w
# dd if=F_TS-569_20120628-1.2.2.img bs=512 of=part6 skip=990240 count=8688w  
... which gives  
-rw-r--r-- 1 root   root     2211840 2013-08-30 15:41 part1
-rw-r--r-- 1 root   root   248119296 2013-08-30 15:42 part2
-rw-r--r-- 1 root   root   248119296 2013-08-30 15:42 part3
-rw-r--r-- 1 root   root     8503296 2013-08-30 15:42 part5
-rw-r--r-- 1 root   root     8896512 2013-08-30 15:42 part6

Partition 3 is a mirror of partition 2, verified through md5sum. Partitions 5 and 6 are empty, likely for scratch space. Partition 1 is /boot/grub which contains modules and the like for booting and hardware configuration. So lets look at partition 2, the boot partition.

# mkdir /mnt/ts2
# mount -r part2 /mnt/ts2 -o loop
# ls -la /mnt/ts2/boot
-rw-r--r-- 1 root root  3982976 2012-06-27 22:17 bzImage
-rw-r--r-- 1 root root       81 2012-06-27 22:17 bzImage.cksum
-rw-r--r-- 1 root root  8890727 2012-06-27 22:17 initrd.boot
-rw-r--r-- 1 root root       85 2012-06-27 22:17 initrd.boot.cksum
-rw-r--r-- 1 root root 73175040 2012-06-27 22:17 qpkg.tar
-rw-r--r-- 1 root root       83 2012-06-27 22:17 qpkg.tar.cksum
-rw-r--r-- 1 root root 33593992 2012-06-27 22:17 rootfs2.bz
-rw-r--r-- 1 root root       85 2012-06-27 22:17 rootfs2.bz.cksum
-rw-r--r-- 1 root root 31160679 2012-06-27 22:17 rootfs_ext.tgz
-rw-r--r-- 1 root root       87 2012-06-27 22:17 rootfs_ext.tgz.cksum
# file -z /mnt/ts2/boot/initrd.boot
/mnt/ts2/boot/initrd.boot: Linux rev 1.0 ext2 filesystem data, UUID=770ce31c-d03f-484e-81e8-6911340bdcbf (gzip compressed data, from Unix, last modified: Wed Jun 27 22:16:58 2012, max compression)

• bzImage is the compressed kernel image
• initrd is the initial ramdisk root filesystem that gets the OS running
• qpkg.tar holds various software packages for the NAS
• rootfs2.bz is a compressed tarball of some /home, /lib, and /usr files
• rootfs_ext.tgz is a compressed tarball of another ext2 filesystem of /opt/source for apache, php5, mysql, and what appears to be a backup of the NVRAM settings.

All of the magic is inside the initrd filesystem image. Peering into that we get:

# gunzip -c /mnt/ts2/boot/initrd.boot >/tmp/initrd.boot.img
# mkdir /mnt/tsinitrd
# mount -r /tmp/initrd.boot.img /mnt/tsinitrd -o loop
# ls -la /mnt/tsinitrd
drwxr-xr-x  2 root root  2048 2012-06-27 22:05 bin
drwxr-xr-x  5 root root 13312 2012-06-27 22:11 dev
drwxr-xr-x 22 root root  2048 2012-06-27 22:15 etc
drwxr-xr-x  3 root root  3072 2012-06-27 22:05 lib
drwxr-xr-x  2 root root  1024 2010-11-03 04:53 lib64
lrwxrwxrwx  1 root root    11 2012-06-27 22:16 linuxrc -> bin/busybox
drwx------  2 root root 12288 2012-06-27 22:16 lost+found
drwxr-xr-x  4 root root  1024 2012-06-27 22:04 mnt
drwxr-sr-x  2 root root  1024 2012-06-27 22:16 opt
lrwxrwxrwx  1 root root    19 2012-06-27 22:16 php.ini -> /etc/config/php.ini
drwxr-sr-x  2 root root  1024 1999-11-02 18:54 proc
lrwxrwxrwx  1 root root    18 2012-06-27 22:16 Qmultimedia -> /share/Qmultimedia
drwxr-xr-x  3 root root  1024 2007-07-18 05:24 root
drwxr-xr-x  2 root root  5120 2012-06-27 22:15 sbin
drwxrwxr-x 29 root root  1024 2006-02-28 00:57 share
drwxrwxrwx  4 root root  1024 2006-02-28 00:57 tmp
drwxrwxrwx  8 root root  1024 2012-06-27 22:15 var

Remember the 2 things that stuck out from the Firmware Recovery page? The update script and the encryption reference:

# more /mnt/tsinitrd/etc/init.d/update.sh
...
... line 223
    /sbin/PC1 d QNAPNASVERSION4 $path_name ${_tgz};
...

There's the reference to what appears to be the encryption key and perhaps the decrypter! Since this NAS firmware image is x86 based, and I'm in an x86 VM, might as well try it:

# /mnt/tsinitrd/sbin/PC1
Usage: pc1 e|d "key" sourcefile <targetfile>
where: e - encrypt, d - decrypt & "key" is the encryption key.
The length of the key will determine strength of encryption
If no targetfile, output file name is equal to sourfile name
ie: 5 characters is 40-bit encryption.

And finally:

# /mnt/tsinitrd/sbin/PC1 d QNAPNASVERSION4 TS-569_20130726-4.0.2.img TS-569_20130726-4.0.2.tgz
Using 120-bit encryption - (QNAPNASVERSION4)
len=1048576
model name = TS-569
version = 4.0.2

# tar -tvf TS-569_20130726-4.0.2.tgz 
-rw-r--r-- root/root       106 2013-07-25 20:49 bios_layout
drwxr-xr-x root/root         0 2013-07-25 20:49 boot/
-rw-r--r-- root/root   4557984 2013-07-25 20:49 bzImage
-rw-r--r-- root/root        69 2013-07-25 20:49 bzImage.cksum
drwxr-xr-x root/root         0 2013-07-25 20:49 config/
-rwxr-xr-x root/root     48408 2013-07-25 20:49 dmidecode
-rwxr-xr-x root/root    356714 2013-07-25 20:49 flashrom
-rw-r--r-- root/root   2097152 2013-07-25 20:49 flashrom.img
-rw-r--r-- root/root        33 2013-07-25 20:49 fw_info
-rw-r--r-- root/root   8480290 2013-07-25 20:49 initrd.boot
-rw-r--r-- root/root        73 2013-07-25 20:49 initrd.boot.cksum
-rwxr-xr-x root/root   1606508 2013-07-25 20:49 libcrypto.so.1.0.0
-rwxr-xr-x root/root    372708 2013-07-25 20:49 libssl.so.1.0.0
-rw-r--r-- root/root  81090560 2013-07-25 20:49 qpkg.tar
-rw-r--r-- root/root        72 2013-07-25 20:49 qpkg.tar.cksum
-rw-r--r-- root/root  41185897 2013-07-25 20:49 rootfs2.bz
-rw-r--r-- root/root        74 2013-07-25 20:49 rootfs2.bz.cksum
-rw-r--r-- root/root  47500086 2013-07-25 20:49 rootfs_ext.tgz
-rw-r--r-- root/root        78 2013-07-25 20:49 rootfs_ext.tgz.cksum
drwxr-xr-x root/root         0 2013-07-25 20:49 update/
-rw-r--r-- root/root       105 2013-07-25 20:49 update_bios.conf
-rwxr-xr-x root/root      3188 2013-07-25 20:49 update_bios.sh
-rwxr-xr-x root/root      6088 2013-07-25 20:49 update_check
-rwxr-xr-x root/root     22041 2013-07-25 20:49 update_img.sh

All of that to get to an executable that decrypts the firmware image for us, a script that gives us the decryption key in plain text, and a way to package everything back together if we wanted to modify something.

... and now for something completely different

The Very Easy Way

Once I got to the end of the "hard way", I decided to google for the encryption key "QNAPNASVERSION4". The first result was for the PC1 enc/dec algorithm in C that someone has already so kindly modified to handle the firmware format specifics for us: http://www.r00ted.com/downloads/pc1.c

Update: Link was reported broken, here's a dump: http://pastebin.com/KHbX85nG

# gcc -o pc1 pc1.c
# pc1 d QNAPNASVERSION4 TS-569_20130726-4.0.2.img TS-569_20130726-4.0.2.tgz
# tar -tvf TS-569_20130726-4.0.2.tgz
-rw-r--r-- root/root       106 2013-07-25 20:49 bios_layout
drwxr-xr-x root/root         0 2013-07-25 20:49 boot/
-rw-r--r-- root/root   4557984 2013-07-25 20:49 bzImage
... same result as the hard way

Now you have a utility that will decrypt your firmware file from the comfort of your own OS without needing physical access to the NAS.

Answer 2

The file indeed looks encrypted or obfuscated. It might be possible to figure it out using some cryptoanalysis (that 34 48 83 sequence at the end doesn't look random), but you'll probably be better off looking into finding UART or JTAG pins, or maybe a running telnet server or another service which may give you a way in.

EDIT: At the download page for the NAS, there are smaller downloads called "Qfix". They seem to be simple self-extracting shell script+tar.gz data. I suggest you try making your own .qfix with a shell script that would copy files off the device instead of the normal behavior.

However, there is the file footer which is probably used for integrity checking. The number next to "SambaFix" looks like some checksum.

Answer 3

I know this is an older question now, but I just want to add some updated information to the discussion for future answer-seekers.

I'm also experimenting with this. Newer versions of binwalk have an automatic extract feature. Simply run:

binwalk -e ./firmwarefile.bin

It will extract and split up the different partitions into separate folders. This is much, much easier than using dd, where if you get the values wrong then squashfs/gzip/... will only see it as corrupted.

Also, grab a copy of Kali or Backtrack linux (Backtrack might be depreciated now. I think, Kali is preferred) They have binwalk and firmware mod utils either installed or in the apt package manager repo for easy install.

As for repackaging a firmware for uploading, I haven't gotten that far yet, myself. With the firmware I am using, binwalk also extracts the file with the signatures, as well. This contains the MD5 data for verifying the firmware on the device.

Answer 4

I found a couple of tools to play with firmwares, but none of them is useful to "play" with firmware I downloaded for my media player, maybe it is more useful to you:

http://www.routertech.org/tools/firmware_tool-097b.zip

firmware mod kit: http://code.google.com/p/firmware-mod-kit/downloads/list

I wanted just to figure out which web pages are stored on the device, which can work as NAS, but I have no access to Linux boxes.