Anyone recognize this variant of Intel HEX firmware format?

Question

I've got a firmware file for an ARM Cortex-M0+ that is in a format extremely similar to Intel HEX. All data is encapsulated in a 0x3a ... 0xa wrapper. The fields seem to mostly match the 16 bit Intel HEX structure, where the second field is definitely the record length, the next two are plausibly an address to load the payload. The next under Intel HEX should be the record type, and I'll cover what I see here in a second.

I see two significant variations from the standard Intel HEX (I have no experience with it, just what I've read)

1. The fields are all specified in raw bytes, NOT ASCII representations of the bytes. So for instance, most records are 16 bytes long and hexdump shows: 0x3a 0x10 ... 0xa
2. The last field before the record delimiter should be the two's compliment of the LSB of the sum of all fields. That is not what I'm seeing, and I don't know how it is being calculated. This is kinda important since I want to modify this firmware. Also, my python code for two's compliment calculation could be wrong, so feel free to tell me I'm just an idiot.

Any ideas what this format might be? Here's a little sample hexdump output:

00000000  3a 10 04 00 00 ff ff ff  ff ff ff ff ff ff ff ff  |:...............|
00000010  ff 7e fb ff ff 81 0a 3a  10 08 00 00 00 60 00 20  |.~.....:.....`. |
00000020  c1 08 00 00 49 93 01 00  4d 93 01 00 e1 0a 3a 10  |....I...M.....:.|
00000030  08 10 00 51 93 01 00 55  93 01 00 59 93 01 00 5d  |...Q...U...Y...]|
00000040  93 01 00 2c 0a 3a 10 08  20 00 61 93 01 00 65 93  |...,.:.. .a...e.|
00000050  01 00 69 93 01 00 6d 93  01 00 dc 0a 3a 10 08 30  |..i...m.....:..0|
00000060  00 71 93 01 00 75 93 01  00 79 93 01 00 1f 2f 02  |.q...u...y..../.|
00000070  00 4d 0a 3a 10 08 40 00  1f 2f 02 00 1f 2f 02 00  |.M.:..@../.../..|

Back to that record type field. Out of a 200kb file, I have only 6 records that are not of type 0:

• Type 1: One instance. Should be EOF, last record, but it's not. It is a record of zero payload length, as it should be, but I have one more record after it, Type 6.
• Type 2: I have three instances. Should be Extended Segment Address, according to wikipedia the address field is typically zero and the data length should be two. That is what I see for all three instances.
• Type 3: Comes just before the Type 1 (EOF?) record. Matches what Wikipedia tells me to expect for Type 3. Looks like: 3a 04 00 00 03 00 00 08 c1 30 0a. Maybe this is the CS:IP that wikipedia says?
• Type 6: Last record, looks like: 3a 04 00 00 06 f9 f7 b1 9f b6 0a.

UPDATE: The checksum field in my file is actually the two's compliment, the original code I wrote was broken in several ways. Dang type-less python code. What eventually worked for calculating the checksum was:

def twos_comp(val, bits):
    return val - (1 << bits)        

def calc_checksum(record_byte_string):
    return abs(twos_comp(sum(record_byte_string)%256, 8))%256

So my only real question left is: what is the Type 6 record at the very end? I'm guessing its a 4-byte CRC over something, but I don't know what. Anyone seen this before?

Answer 1

Since it's using raw bytes and not textual representation, it doesn't seem to be related to Intel HEX or Motorola S-Record at all. It's probably just some custom chunk-based format developed by the manufacturer of the device (possibly inspired by Intel Hex though). I'd suggest you to RE the firmware updater program or the receiving part in the firmware, maybe it will have the code for handling the unknown record.

Answer 2

It turns out that record type 6 is a 32-bit CRC, in bigendian order. The CRC is the typical (as I understand it) CRC32, producing identical results to the crc32 utility included in the Archive::Zip Perl module on Linux systems. The trick was that the bootloader is actually calculating it over a region of flash memory storing the application code. I haven't fully traced the bootloader, but the code appears to be installing the firmware update and only then checking its integrity against this value, after boot. I'm not sure what bootloader it is, but it appears to not be the Kinetis bootloader, given the absence of the magic string required in the Kinetis bootloader Bootloader Configuration Area (BCA).

So the trick was figuring out what the flash would look like after applying the updates specified in the update file, and doing a CRC over only the region of flash the bootloader was calculating it over. In my case that was 0x800 to 0x3afff. Also in my case, the unspecified bytes were set to 0xFF, so I can calculate an offline CRC just by looking at the file. But for one other update file I have, the assumption that unspecified bytes will be 0xFF does not hold, so I guess I just got lucky.

One interesting note: I used the crcmod Python module to try to calculate my CRCs when I was trying to brute-force guess things, when I posted my original question. Even when I knew the CRC algorithm being used, and over what byte range, I still could not get any crcmod CRC algorithm to give me the right result. I fiddled with it for a little while, but eventually gave up and just used the crc32 shell utility. I never got to the bottom of that problem, and it easily could have been a bug in my Python code, but I wanted to just note that for anyone experiencing similar issues.