1

January 17, 2015 update,

We made some progress by converting the CSV to hcc file; however, MT5 is deleting the file. Attached is our code. Can someone help us?

http://www.mediafire.com/download/n5454ao79mmqksf/CSVtoHCC.rar


Can someone give us some pointers as how to crack this hcc file? It took a long time but a friend was able to determine its data structure. See link below for the hcc file.

4 byte, seperator , little endian encoding 18385028, hex(84 88 18 01)

4 byte, time, int divisible by 60

8 byte, double Opened Price

8 byte, double Highest Price

8 byte, double Lowest Price

8 byte, double Closed Price

1 byte, char | small int spread

1 byte, char | small int tick volume

date & time format is: 2015.01.04 0:00

Opened Price = 1.23894 Highest Price = 1.23904 Lowest Price = 1.23883 Closed Price = 1.23904 Tick Volume = 81

2015.01.04 0:00 1.23894 1.23904 1.23883 1.23904 81

Many thanks in advance.

http://www.mediafire.com/download/2csbb4s3b9bjx8e/2015.hcc

  • 3
    I'm not sure what your question is, if you already know (a part of) what your file format is like. But your separators are actually 84 88 18 X1, where X is the number of additional bytes between each of your 'record's and the next one. – Guntram Blohm Jan 13 '15 at 8:02
  • 1
    Looking at your separators a bit more, they're actually byte counts. Your little endian 0x01188884, read from behind, translates to 4 bytes (time), 8 bytes double (4 times), one byte, one byte, zero bytes. At 0xde1a in your file, your separator is 0x42188884, and after your doubles, you have 1 byte int (40), a 2 byte int (333, supposed to be volume), and a 4 byte int, 72000000, (whatever that means). – Guntram Blohm Jan 13 '15 at 15:44
4

The file seems to contain a "sum of transactions" field as well. I refined my earlier program a bit:

/* version 1.2 - generates a CSV file on the desktop for another application to create a reverse engineered hcc file */
/* version 1.1 - includes header format and detects the number of data rows */

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

#pragma pack(1)
union header {
                struct {
                                char magic[2]; // 0x81 0x00
                                char label[64];
                                char something[18];
                                int nrows;
                } data;
                char rawbytes[189];
};

struct record {
                int separator;
                int time;
                double open, high, low, close;
                char additional[10]


};


void printint(char *buf, FILE *fpcsv,int size);
void dumpat(FILE *fp,FILE *fpcsv, int start);

int main(int argc, char **argv) {
                FILE *fp,*fpcsv;
                union header header;


                if ((fpcsv=fopen("C:\\Users\\Hello\\Desktop\\2015.csv", "w"))==NULL) {
                        perror("C:\\Users\\Hello\\Desktop\\2015.csv"); return 1;
                }
                if ((fp=fopen("C:\\Users\\Hello\\Desktop\\2015.hcc", "rb"))==NULL) {
                        perror("C:\\Users\\Hello\\Desktop\\2015.hcc"); return 1;
                }
                fseek(fp, 0x3be5-sizeof(header), 0);
                while (fread(&header, sizeof header, 1, fp)!=0) {
                                if ((header.data.magic[0]&0xff)!=0x81
                                ||   header.data.magic[1]!=0x00) {
                                                fprintf(stderr, "Bad header magic number at %08lx\n",
                                                                ftell(fp)-sizeof(header));
                                                exit(1);
                                }
                                fprintf(fpcsv,"Expect %d rows\n", header.data.nrows);
                                dumpat(fp,fpcsv, header.data.nrows);
                // dumpat(fp, 0x3be5);
                // dumpat(fp, 0xde1a);
                // dumpat(fp, 0x1e169);
                // dumpat(fp, 0x2e4a5);
                // dumpat(fp, 0x3bd7e);
                // dumpat(fp, 0x55c07);
                }
                fclose(fp);
                fclose(fpcsv);
}

void dumpat(FILE *fp,FILE *fpcsv, int nrows) {
                struct record data;
                int i, extra1, extra2, extra3, pos;
                time_t time;
                struct tm *tm;
                char timebuf[80];

                for (i=0; i<nrows; i++) {
                                if (fread(&data, sizeof(int), 1, fp)==0)
                                                break;
                                if ((data.separator & 0x00088884) != 0x00088884) {
                                                fprintf(stdout, "bad separator %08x at %lx\n",
                                                                data.separator, ftell(fp));
                                                break;
                                }
                                extra1=data.separator>>28;
                                extra2=((data.separator>>24)&0x0f);
                                extra3=((data.separator>>20)&0x0f);
                                fread(&data.time,  4 + 8 + 8 + 8 + 8 + extra3 + extra2 + extra1
                                                , 1, fp);
                                time=data.time;
                                tm=gmtime(&time);
                                strftime(timebuf, sizeof timebuf, "%Y-%m-%d %H:%M:%S", tm);
                                fprintf(fpcsv,"%s,%lf,%lf,%lf,%lf,", timebuf, data.open,
                                                data.high, data.low, data.close);
                                pos=0;
                                printint(data.additional+pos,fpcsv, extra3); pos+=extra3;
                                fprintf(fpcsv,",");
                                printint(data.additional+pos,fpcsv, extra2); pos+=extra2;
                                fprintf(fpcsv,",");
                                printint(data.additional+pos,fpcsv, extra1); pos+=extra1;
                                fprintf(fpcsv,"\n");

                }
}

void printint(char *buf,FILE *fpcsv, int size) {
                int val=0, scale=1;
                while (size--) {
                                val+=(*buf++&0xff)*scale;
                                scale<<=8;
                }
                 fprintf(fpcsv,"%5d", val);
}

The result of this begins with:

2015-01-02 10:00:00 1.205450 1.205450 1.205140 1.205160    31     9 22000000 
2015-01-02 10:01:00 1.205160 1.205210 1.205030 1.205030    31     6 35000000 
2015-01-02 10:02:00 1.205030 1.205030 1.204680 1.204730    51     9 54500000 
2015-01-02 10:03:00 1.204730 1.204750 1.203460 1.203850    81    18 65700000 
2015-01-02 10:04:00 1.203790 1.204210 1.203750 1.204210    79     9 85100000 
2015-01-02 10:05:00 1.204200 1.204210 1.204090 1.204170    41    10 38300000 
2015-01-02 10:06:00 1.204170 1.204570 1.204150 1.204150    52    13 46600000 
2015-01-02 10:07:00 1.204150 1.204460 1.204150 1.204380    44     8 53900000 
2015-01-02 10:08:00 1.204380 1.204380 1.203800 1.203800    52     9 70400000 
2015-01-02 10:09:00 1.203820 1.204010 1.203600 1.203900    72     9 76100000 

The headers (that contain EUR USD) seem to have the same size, and the start of the data seems to be 176 bytes after the D every time, but i'm not yet sure about where/how to find the first of them, and where the number of data rows to the next header is stored, so i can't get rid of those magic numbers (byte positions where the data starts) yet.

  • Guntram,Thank you very much for helping us out. Attached are three different hcc files from three different brokers. I believe that each broker will have its own identifier. mediafire.com/download/9d0ux9uh74lpd2c/… – Uncle Bob Jan 14 '15 at 5:48
  • Seems the first block of data is always at 0x3be5, the headers have always 189 bytes and start with 0x81 0x00 and are followed by the EURUSD string, and there's the number of data rows at offset 84 in the header. Download the new program at mega.co.nz/…. – Guntram Blohm Jan 14 '15 at 10:38
  • Guntram. Just to let you know that we are working on the codes at the moment and will post them up once finish. Code #1 will slightly modify your version 1.1 (i.e. version 1.2) and will generate a CSV file on the desktop. Code#2 will import the CSV file from the desktop and then attempt to generate reverse engineered hcc file. – Uncle Bob Jan 14 '15 at 17:15
  • attached is version 1.2, please let us know how we can improve it further mediafire.com/view/zsuoxj4h4506s5l/main.c – Uncle Bob Jan 14 '15 at 21:37
  • Here is something interesting. I found another broker using the same trading platform; however, the file has a hcs instead of hcc extension. mediafire.com/download/itp8h3462bq2weo/2015.hcs – Uncle Bob Jan 14 '15 at 22:48
0

It would help to have the output of this actual file. After a quick search, I found:

$ strings -e l 2015.hcc
Copyright 2001-2014, MetaQuotes Software Corp.
History
EURUSD
EURUSD
EURUSD
EURUSD
EURUSD
EURUSD

Which lead to the following website. What's in this file ? Using clear text attack would be much more efficient. Do you have more sample files ?

Using binwalk did not reveal anything, so I am guessing this is a simple binary format, since it contains -potentially- trading information I would bet this is a simple format (no cryptography or obfuscation involved).

  • The OP documents most of the file format. The file actually has a simple binary format, and from the OP, it's possible to write a simple c program to read the data. This is just why i don't really understand OPs question. – Guntram Blohm Jan 13 '15 at 15:18
  • The HCC file contains the following information (see attached file link) plus some additional unknown information mediafire.com/view/60tww4c2tvj6b5f/data_2.csv – Uncle Bob Jan 13 '15 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.