I have to decode a proprietary file format which mainly contains time series data. The only thing I was told was that the content is 'zipped', although I am not sure whether it is really zipped or some in-house compression method is used. Here is a snippet of the data:


I did a [byte] frequency analysis and it shows a uniform distribution. If the bytes are plot (with A.X.E.), a very homogeneous image is shown:

enter image description here

I've checked for the header of common compression algorithms but found none of them.

Any help is highly appreciated.


Some of these file can be accessed on the following links: First file, Second file, Third file, Forth file

The executable which produced/validated these files is not available.


Using binwalk -Me <filename> (github.com/devttys0/binwalk) it is possible to extract two zlib-ed files from each of the previous files.

Here's the header of one of the files:

  • Is the code generating or processing the file available to you? how many file samples can you collect? are you able to trigger any validation over modified files? can you share the entire file? more info would help here
    – NirIzr
    Sep 21 '16 at 10:49
  • sharing the entire file would help immensely. also,what is there once you decompress the zlib'd blobs?
    – grepNstepN
    Sep 21 '16 at 14:38
  • Best way to analyse proprietary file format - is to analyze software that should receive and process this file format. I think you should start with RE of binary that able to create file with this file format, read it or convert from\to it. You have access to those? Sep 22 '16 at 11:20
  • @DenisLaskov OP already mentioned he does not. "The executable which produced/validated these files is not available." izibe it would be nice of you to upload the uncompressed files, now that you've figured out the first layer.
    – NirIzr
    Sep 22 '16 at 19:39

It would help if I knew what the data represented, but I have determined the format of the files once unzipped. Each unzipped file has this structure:

short unk1;           // 10
short num_records;    // 8751  (8760 hours per year)
short unk2;           // 0
short num_samples;    // varies per file from 8 to 179
short unk3;           // 0
short unk4;           // 40
short unk5;           // 1000
short unk6;           // 100
int data_bytecount;   // varies per file (remaining byte count of the file)
Record records[num_records];

Where each Record is defined as: (defined as an 010Editor template)

typedef struct {
    byte flag;                // 0 = no data; 1 = sample data follows
    if(flag == 1)
        short data[num_samples];
} Record;

For example, in the first file unzipped from "295", there are 8751 records with 179 samples each. Most records only have ~30 non-zero values. The samples for records[0] look like:

15185, 15185, 15060, 14968, 14719, 14583, ...

File#1 from "295", Record 0

I suspect the data values are normalized floats, perhaps dividing them by 1000.0 or using 16384 as a zero reference for positive/negative numbers. I also suspect each file contains 1 year's worth of records with 1 per hour.

I'd be interested in knowing what the values represent ... just to have a complete picture.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.