3

When reversing a router firmware based on DD-WRT I came across a mention of "webcomp". It seems it's used for storing internal HTML files for the web interface. What exactly it is and how can I extract those files from the firmware image?

NOTE: This is a self-asked and answered question to add to the knowledge based.

  • Is this format really the way to go? Looks more like something that would belong on a wiki describing webcomp not as a QA. – Peter Andersson Apr 14 '13 at 12:52
  • 1
    @90h: chime in over on meta then ;) – 0xC0000022L Apr 14 '13 at 13:26
  • @90h Votes on meta are an expression of opinion. In any real world scenario, there will be people who agree with you and there will be those who dont. You should not let that stop that stop you from expressing your opinion about how this site should work. – asheeshr Apr 14 '13 at 16:39
  • It doesn't sound like a real 'pinpointed' question to me, albeit interesting, so I copied the answer/write-up in the new webcomp tag wiki as well. – Ange Apr 14 '13 at 19:26
  • 2
    @90h while I like your enthusiasm I think the way you've been asking these questions is somewhat off. I edited the question a bit to make it a bit more like a real one. – Igor Skochinsky Apr 14 '13 at 20:31
6
/*
 * webcomp -- Compile web pages into C source
 *
 * Copyright (c) GoAhead Software Inc., 1995-2000. All Rights Reserved.
 */

Webcomp Description

Webcomp, or 'web compiler', is a very simple tool written by GoAhead Software during the period of 1995-2000. It 'compiles' web pages into a single binary blob, adding an index of files in the blob to the httpd for use at runtime.

Webcomp is available in the DD-WRT source code. It clearly was created in a 'just hack it out' style, without much concern for its appearance. It does the job though, which is what matters.

DD-WRT uses this tool to 'protect' their web management console. Since their web management console is derived from GPL'd open source, has had countless contributors over the years, and webcomp itself is open source, I do not believe there is any legal restriction against documenting the mechanism they use.

Data Types

Binary Blob. Normally stored as a file at /etc/www, this is a simple concatenation of all web pages on the device. It is not compressed or obfuscated. Nor should it be. The file systems used by embedded linux firmwares, such as squashfs, feature compression as a primary attribute. Therefore, compressing this blob would be redundant and would harm the final compression ratio of the file system. Similarly, obfuscation of the block would substantially affect the compressibility. However, there is a commented out section in the webcomp source that would apply lzma to the binary blob. There's no evidence it has ever been used, for the reasons I've given.

Index Array. The array added to the httpd is done by modifying the source code prior to build. It adds an object named websRomPageIndex that contains this array of information about the files in the concatenated binary blob.

The array consists of file names and sizes. An older version of webcomp included offsets as well, but these are redundant since the binary blob's files are concatenated in the same order as they appear in the index array. Therefore, as one walks the array, the size of each file gives a relative offset to the next file in the binary blob.

Craig Heffner's webdecomp, which can decompile and recompile webcomp'd web packages, is much better formatted code than webcomp itself, so I'll use its data structures, amended with my own comments.

Each member of the array is defined as:

ORIGINAL Format

/* OLD webcomp file_entry data type */
struct file_entry
{
    uint32_t name;   /* virtual offset of asciiz file name in httpd  */
    uint32_t offset; /* offset to file in blob (unnecessary) */
    uint32_t size;   /* size of file data in blob */
};

NEW Format

/* NEWER webcomp file_entry data type */
struct new_file_entry
{
    uint32_t name;  /* virtual offset of asciiz file name in httpd  */
    uint32_t size;  /* size of file data in blob */
};

Modifications by DD-WRT

DD-WRT has hacked on the webcomp code a few times. Their modifications are not publicly available. In fact, only the old webcomp.c using the original file_entry data type is included in the DD-WRT source code. They may have been responsible for the switch to the more efficient file_entry data type that excludes the redundant offsets.

They are certainly responsible for a recent addition that obfuscates all file sizes in the index array. They do this by adding a constant to the sizes. In the most recent builds of DD-WRT, this appears to be 0x4BF8. This can vary between builds.

Therefore, the obfuscation is quite simple:

new_file_entry.size += key;

Dynamically Calculating the Key. Since we can assume the real size of files within the package (many haven't changed in years, if ever), we can calculate the key used with a firmware. This has the added benefit of working on older images that had no key (0). We do this by finding the file we know the size of (index 0 is fine), then:

key = new_file_entry.size - assumed_known_size;

For an additional sanity check, one could confirm that the sample file is indeed extracted to match the known sample of that file. A simple hash compare would do the trick.

Update: See devttys0's answer that utilizes the sum of all files vs the actual blob size, then divides by the number of files to get the average difference, which is the key.

Webdecomp

Created by Craig Heffner, this tool can extract and rebuild webcomp'd packages. It's, ironically, far superior code to the webcomp tool. I recently modified webdecomp to handle these new DD-WRT keys, calculating the key at extraction and saving it for the Later rebuild. This is part of the Firmware Mod Kit.

4

Small update to 90h's answer: the key can be calculated dynamically without making assumptions about the size of any specific file.

The total of all the file sizes reported in httpd should match the size of the /etc/www file, which is the actual size of all the files. Since the same fixed value is added to all the file sizes reported in httpd, you can simply take the difference in reported size vs actual size and divide it by the number of files to find what fixed value was used:

key = (total_reported_size - actual_etc_www_size) / number_of_files
-1

The webdecomp tool didn't work for me on the latest builds of DD-WRT, managed to extract the firmware though.

root@ubuntu:~/Desktop/firmware-mod-kit/src/webcomp-tools$ ./webdecomp --httpd="/home/root/Desktop/firmware-mod-kit/fmk/rootfs/usr/sbin/httpd" -www="/home/root/Desktop/firmware-mod-kit/fmk/rootfs/etc/www" --dir="/home/root/Desktop/www" --extract
Failed to locate websRomPageIndex!
Failed to detect httpd settings!
Failed to process Web files!

root@ubuntu:~/Desktop/firmware-mod-kit/fmk/rootfs/usr/sbin$ file httpd 
httpd: ELF 32-bit LSB executable, MIPS, MIPS32 version 1, dynamically linked, interpreter /lib/ld-uClibc.so.0, corrupted section header size

Looks like they have obfuscated the section headers.

root@ubuntu:~/Desktop/firmware-mod-kit/src/webcomp-tools$ readelf -a /home/root/Desktop/firmware-mod-kit/fmk/rootfs/usr/sbin/httpd
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 01 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       1
  Type:                              EXEC (Executable file)
  Machine:                           MIPS R3000
  Version:                           0x1
  Entry point address:               0x401a30
  Start of program headers:          52 (bytes into file)
  Start of section headers:          0 (bytes into file)
  Flags:                             0x54001005, noreorder, cpic, o32, mips16, mips32
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         7
  Size of section headers:           0 (bytes)
  Number of section headers:         0
  Section header string table index: 0

There are no sections in this file.

There are no sections to group in this file.
  • 1
    This is not an answer to the posted question. Feel free to post a new question (check the rules first); you can link to this one for context. – Igor Skochinsky Sep 23 '18 at 19:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.