Looking at it in OllyDbg it looks like a heavy task. Looks like a custom database with encrypted and (custom?) compressed data. This or the like would usually be the case in such applications. A flat file with structured data is not part of this one.
Anyhow. As a starter:
A quick check after trying out some general compression tools like 7z or binwalk, (have not tested it), can be to use ProcMon from Sysinternals. Start ProcMon, then your application and set filter on the application in ProcMon. You quickly find that:
In short it reads in chunks of varying size, but for main data processing it reads chunks of 16384 bytes. The process in steps:
- Generate seed map of 256 integers. (Done once at application start.)
- Loop:
2.1 Read 16384 bytes into buffer from .dat file.
2.2 XOR routine on buffer using offset and last four bytes of buffer as base.
2.3 Checksum on XOR'ed buffer using seed map from step 1.
2.4 Parse buffer and read out data.
The application also reads same chunks multiple times.
2.1:
Example:
013D0010 D4 9E BE BF 1C 1C 0B D4 C5 E7 11 B5 09 48 87 FA Ôž¾¿ÔÅçµ.H‡ú
013D0020 29 4C 03 C9 DE 4A 2B 71 74 7F D2 48 E7 13 94 4E )LÉÞJ+qtÒHç”N
...
013D3FF0 6A D1 55 92 E2 16 60 53 69 89 86 7D D9 D8 10 BC jÑU’â`Si‰†}Ùؼ
013D4000 90 F3 D1 48 28 47 34 EC 39 36 EC 4D 69 2A 7D E5 óÑH(G4ì96ìMi*}å
|_____._____|
|
Last DWORD aka checksum --+
Steps and details in order of discovery:
Split the .dat file in chunks of 16384 bytes and also generate a hex-dump of each file for easy search and comparison. To be honest I use Linux for this part with dd, xxd -ps, grep, diff etc.
Start OllyDbg, open the application, locate CreateFile and set breakpoint:
00401220 $-FF25 18825000 JMP DWORD PTR DS:[<&kernel32.CreateFileA>; kernel32.CreateFileA
Press F9 until filename (in EAX) is .dat file. Set/enable breakpoint on ReadFile. F9 and when read is done start stepping and looking at what is done.
Looking at it:
2.2:
After read it first modify the buffer by using offset as "magic" starting at:
0045F5EC /$ 53 PUSH EBX ; ALGO 2: XOR algorithm - post file read.
...
0045F6B6 \. C3 RETN ; ALGO 2: RETN
At least two of the actions taken seems to be libj_randl1() and libj_randl2(). (This would be step 2.2 in list above.)
Simplified:
edx = memory address of buffer
ecx = offset / 0x4000
edi = edx
ebx = ecx * 0x9b9
esi = last dword of buffer & 0x7fffffff
ecx = 0
i = 0;
while (i < 0x3ffc) { /* size of buffer - 4 */
manipulate buffer
}
The whole routine translated to C code:
int xor_buf(uint8_t *buf, long offset, long buf_size)
{
int32_t eax;
int32_t ebx;
int32_t esi;
long i;
buf_size -= 4;
ebx = (offset / 0x4000) * 0x9b9;
/* Intel int 32 */
esi = (
(buf[buf_size + 3] << 24) |
(buf[buf_size + 2] << 16) |
(buf[buf_size + 1] << 8) |
buf[buf_size + 0]
) & 0x7fffffff;
for (i = 0; i < buf_size /*0x3ffc*/; ++i) {
/* libj_randl2(sn) Ref. link above. */
ebx = ((ebx % 0x0d1a4) * 0x9c4e) - ((ebx / 0x0d1a4) * 0x2fb3);
if (ebx < 0) {
ebx += 0x7fffffab;
}
/* libj_randl1(sn) Ref. link above. */
esi = ((esi % 0x0ce26) * 0x9ef4) - ((esi / 0x0ce26) * 0x0ecf);
if (esi < 0) {
esi += 0x7fffff07;
}
eax = ebx - 0x7fffffab + esi;
if (eax < 1) {
eax += 0x7fffffaa;
}
/* Modify three next bytes. */
buf[i] ^= (eax >> 0x03) & 0xff;
if (++i <= buf_size) {
buf[i] ^= (eax >> 0x0d) & 0xff;
}
if (++i <= buf_size) {
buf[i] ^= (eax >> 0x17) & 0xff;
}
}
return 0;
}
Then a checksum is generated of the resulting buffer, (minus last dword), and checked against last dword. Here it uses a buffer from BSS segment that is generated upon startup, step 1. from list above. (Offset 0x00505000 + 0x894 and using a region of 4 * 0x100 as it is 256 32 bit integers). This seed map seems to be constant (never re-generated / changed) and can be skipped if one do not want to validate the buffer.
1.
Code point in disassembly (Comments mine.):
0045E614 . 53 PUSH EBX ; ALGO 1: GENERATE CHECKSUM MAGICK BSS
...
0045E672 . C3 RETN ; ALGO 1: RETN
The code for the BSS numbers can simplified be written in C as e.g.:
int eax; /* INT NR 1, next generated number to save */
int i, j;
unsigned int bss[0x100] = {0}; /* offset 00505894 */
for (i = 0; i < 0x100; ++i) {
eax = i << 0x18;
for (j = 0; j < 8; ++j) {
if (eax & 0x80000000) {
eax = (eax + eax) ^ 0x4c11db7;
} else {
eax <<= 1;
}
}
bss[i] = eax;
}
2.3:
That bss int array is used on the manipulated buffer to generate a checksum that should be equal to the last integer in the 16384 bytes read from file. (Last dword, the one skipped in checksum routine and XOR'ing.). This would be step 2.3 in list above.
unsigned char *buf = manipulated file buffer;
unsigned char *bss = memory dump 0x00505894 - 0x00505C90, or from code above
eax = 0x13d0010; /* Memory location for buffer. */
edx = 0x3ffc; /* Size of buffer - 4 bytes (checksum). */
...
At exit ecx is equal to checksum.
Code point in disassembly (Comments mine.):
0045E5A8 /$ 53 PUSH EBX ; ALGO 3: CALCULATE CHECKSUM AFTER ALGORITHM 2
...
0045E5E0 \. C3 RETN ; ALGO 3: RETN (EAX=CHECKSUM == BUFFER LAST 4 BYTES)
Shortened to a C routine it could be something like:
int32_t checksum(int32_t map[0x100], uint8_t *buf, long len)
{
int i;
int32_t k, cs = 0;
for (i = 0; i < len; ++i) {
k = (cs >> 0x18) & 0xff;
cs = map[buf[i] ^ k] ^ (cs << 0x08);
}
return cs;
}
It is checked to be OK and then checksum in buffer is set as: two least significant bytes = 0, two most significant bytes are set to some number (chunk number in file or read number, (starting from 0)).
0045F9BF . C680 FC3F0000 >MOV BYTE PTR DS:[EAX+3FFC],0 ; Set two lower bytes of checksum in dat buf to 0
0045F9C6 . C680 FD3F0000 >MOV BYTE PTR DS:[EAX+3FFD],0 ; follows previous
0045F9CD . 66:8B4D F8 MOV CX,WORD PTR SS:[EBP-8] ; Set CX to stack pointer value of addr EBP - 8 (counter of sorts)
0045F9D1 . 66:8988 FE3F00>MOV WORD PTR DS:[EAX+3FFE],CX ; Set .dat buffer higher bytes like CX.
Now after all this is done the actual copying of data starts with even more algorithms. Here the real work starts. Identifying data types, structures, where and what etc. Found some routines that extracted names etc. But everything being Finnish didn't help on making it easier to grasp ;).
The data above could be a start.
Some breakpoints that might be of interest to begin with:
Breakpoints
Address Module Active Disassembly Comment
0045E5A8 Kirjaus5 Disabled PUSH EBX ALGO 3: CALCULATE CHECKSUM AFTER ALGORITHM 2
0045E5E0 Kirjaus5 Disabled RETN ALGO 3: RETN (EAX=CHECKSUM == BUFFER LAST 4 BYTES)
0045E614 Kirjaus5 Disabled PUSH EBX ALGO 1: GENERATE CHECKSUM MAGIC BSS
0045E672 Kirjaus5 Disabled RETN ALGO 1: RETN
0045F5EC Kirjaus5 Disabled PUSH EBX ALGO 2: FILE POST XOR READ ALGORITHM
0045F6B6 Kirjaus5 Disabled RETN ALGO 2: RETN
Some notes:
Keep a backup of the .dat file you are working with. If you abort the application the file often gets corrupted as it, as noted by @blabb, write data back to file. The .dat file also seem to be live so a new download of it would result in different data.
