What compression type has been used here?

Question

I am still working on the binary image from the last question, and have already found out that my "corrupted" data is actually just compressed. But now I have another issue because I don't know what compression type was used.

What I already know:

The data consists out of at least 9 Bytes. The first byte is always a "flag" wich describes in it's 8 bits wehter the next 8 Bytes are compressed or not. A Compressed "Byte" is stored as two Bytes.

Example:

ß\DB\cóA\c
ß -> DF -> 11111011 (MSB) -> 6th byte encoded

This means that "óA" actually stands for "ache" but i have not yet found where "ache" is actually stored in memory.

The Image in question is available here: https://we.tl/t-tpG9EjpSbr

Answer 1

Basics

I've had look at your image file and, as the answer to your previous question said, the compression is something like LZSS.

Specifically, the compressed data begins with a flag byte. Each bit of this in turn (from bit 0 to bit 7) indicates that the next decompressed byte(s) are generated by -

• (when the flag bit is 1) copying a single literal byte from the compressed data stream or

• (when the flag bit is 0) copying between 3 and 18 bytes from earlier in the decompressed data stream. The length and location of these bytes are encoded in 2 bytes in the compressed data stream.

  Suppose these 2 bytes are (in hex using dummy characters) 0xPQ and 0xRS

  Then

  • the length of the data to copy is 3 + 0xS
  • the offset of the data to copy is 18 + 0xRPQ (using unsigned integer arithmetic modulo 0x1000)
  • the offsets appear to be into a 4k circular buffer of the most recently decompressed data

Example

Let's look at decompressing the data at offset 0x0008D38E in the image to storage at address 0x0006E720 (assuming the earlier data has already been decompressed.)

compressed data                                              decompressed data

0008D68E: 7D (flag byte 0b01111101)
             45                        literal byte          0x0006E720: 45
             D2 60                     3 bytes from 0x6E4    0x0006E721: 6D 65 72   (i.e. copied from 0x0006E6E4)
             67 65 6E 63 79            literal bytes         0x0006E724: 67 65 6E 63 79 
             DA 50                     3 bytes from 0x5EC    0x0006E729: 20 4C 6F  (i.e. copied from 0x0006E5EC)

0008D699: FF (flag byte 0b11111111)
             6F 70 20 54 65 73 74 20   literal bytes         0x0006E72C: 6F 70 20 54 65 73 74 20

0008D6A2: 33 (flag byte 0b00110011)
             4F 4E                     literal bytes         0x0006E734: 4F 4E
             08 7F                     18 bytes from 0x71A   0x0006E736: 0D 0A 00 00 00 00 45 6D 65 72 67 65 6E 63 79 20 4C 6F 
             1A 76                     9 bytes from 0x72C    0x0006E748: 6F 70 20 54 65 73 74 20 4F (i.e. copied from 0x0006E72C)
             46 46                     literal bytes         0x0006E751: 46 46
             65 22                     4 bytes from 0x277    0x0006E753: 0D 0A 00 00 00
             06 35                     8 bytes from 0x318    0x0006E758: 45 30 34 0D 0A 00 00 00 

Looking at the decompressed data we see -

0x0006E720: "Emergency Loop Test ON\r\n"
0x0006E73C: "Emergency Loop Test OFF\r\n"
0x0006E758: "E04\r\n"

File Structure

It appears that the whole file is not compressed but contains a couple of compressed regions beginning at offsets 0x00010000 and 0x00040000.
Each compressed region appears to begin with a 4 byte/32 bit value containing the length of the compressed data. These lengths are followed by the compressed data as described above.

Bonus Note

An image search for the name of the device in question along with the 'main board' produces a result with a clearly visible Renesas R8A77240D500BGY. This is an MCU in the Renesas SH7724 family containing a SH-4A RISC CPU core.