-1

I am new in this area.

Scenerio: I am given two files and told one is ASCII and another is binary format of that ASCII file. I opened ASCII file and saw like-

0
76
1.040000
1
1
1.000000
514039
5058933
514808
5059861
2 ********* TIMESTEP
0.750000
3
1
328882
1
514712.000000
5059091.500000
514714.062500
5059088.500000
3.826391
2.000000
13.882331
0.000000
2 ********* TIMESTEP
1.500000
..........................................

And open corresponding binary file (hex)

00 4C B8 1E 85 3F 01 01 00 00 80 3F F7 D7 07 00 75 31 4D 00 F8 DA 07 00 15 35 4D 00 02 00 00 40 3F 03 01 00 00 00 B2 04 05 00 01 00 53 FB 48 27 64 9A 4A 42 53 FB 48 21 64 9A 4A 98 E3 74 40 00 00 00 40 07 1E 5E 41 00 00 00 00 02 00 00 C0 3F 03 01 00 00 00 B2 04 05 00 01 4C 52 FB 48 39 64 9A 4A 8E 52 FB 48 33 64 9A 4A 98 E3 74 40 00 00 00 40 07 1E 5E 41 00 00 00 00 03 02 00 00 00 D2 05 05 00 01 2D 2D FB 48 06 63 9A 4A E2 2C FB 48 FF 62 9A 4A 73 6F 82 40 00 00 00 40 7D 1A 6D 41 00 00 00 00 03 04 00 00 00 BE 04 05 00 01 57 00 FB 48 49 67 9A 4A DE FF FA 48 4D 67 9A 4A 21 D2 86 40 00 00 00 40 0F A4 8C 41 00 00 00 00 02 00 00 10 40 03 01 00 00 00 52 1D 05 00 01 9A 51 FB 48 4A 64 9A 4A DC 51 FB 48 44 64 9A 4A 98 E3 74 40 00 00 00 40 0B 84 5B 41 AB FE 5D BE 03 02 00 00 00 D2 05 05 00 01 FB 2D FB 48 18 63 9A 4A AF 2D FB 48 11 63 9A 4A 73 6F 82 40 00 00 00 40 7D 1A 6D 41 00 00 00 00 03 04 00 00 00 BE 04 05 00 01 D1 01 FB 48 3D 67 9A 4A 58 01 FB 48 41 67 9A 4A 21 D2 86 40 00 00 00 40 0F A4 8C 41 00 00 00 00 03 05 00 00 00 B2 04 05 00 01 06 53 FB 48 27 64 9A 4A 4B 53 FB 48 20 64 9A 4A D5 06 7E 40 00 00 00 40 07 1E 5E 41 00 00 00 00 02 00 00 40 40
.....................................................

Problem-

Is there any way to find how this binary file is crated from that ascii?

I know all of ASCII file what reading means what from a pdf file that tells all about ASCII file.

N.B. I truncated and posted from the larges file as i wish - i mean the trucated portion of ascii and that of binary posted here may not intersect/exact replica. I just copied portion and posted here.

ASCII file link

Binary file link

Thanks

3
  • 1
    Please provide a hex dump of the binary file. (use hexdump -C filename if you use linux).
    – w s
    Dec 7 '15 at 13:51
  • OK edited question.
    – SIslam
    Dec 7 '15 at 14:13
  • if you look close at the two versions, you will see the pattern: first byte in hex is the first byte in ASCII, the next byte in hex is the second byte in ASCII, the the next dword is the floating-point version of the ASCII... for each of the ASCII floating-point values, use a debugger to convert and you'll find the position in the binary. The other fields will become clear. Dec 7 '15 at 16:35
2

As far as I can see it is not encrypted, but just wrote down in binary form straight forward, at least some first fields are exact match.

00              in hexdump means the first zero
4c              in hexdump is 76
B8 1E 85 3F     is binary representation of float 1.039999999 which means 1.04
01              is 1
01              is next 1
00 00 80 3f     is binary representation of 1.0
F7 D7 07 00     is binary representation of integer 514039

and so on.

I used python and python module struct to verify all this.

for example, the following code will give you desired output:

import binascii
import struct

print struct.unpack("b", binascii.unhexlify("00"))
print struct.unpack("b", binascii.unhexlify("4c"))
print struct.unpack("f", binascii.unhexlify("b81e853f"))
print struct.unpack("b", binascii.unhexlify("01"))
print struct.unpack("b", binascii.unhexlify("01"))
print struct.unpack("f", binascii.unhexlify("0000803f"))
print struct.unpack("I", binascii.unhexlify("F7D70700"))

The main problem here is determining the basic sizes of different data. Here is rule of thumb you can use:

  1. Integers may be one byte, 2, 4 or 8 bytes long
  2. floating point values here always were 4 byte long, but also may be 8

All the rest of the work should be obvious.

Transformation back to binary will look as follows:

s = ""

s += struct.pack("b", 0)
s += struct.pack("b", 76)
s += struct.pack("f", 1.04)
s += struct.pack("b", 1)
s += struct.pack("b", 1)
s += struct.pack("f", 1.0)
s += struct.pack("I", 514039)
s += struct.pack("I", 5058933)
s += struct.pack("I", 514808)
s += struct.pack("I", 5059861)

print binascii.hexlify(s)

Here is the code you can start to work from. The code does not produce the file that exactly matches the binary because sometimes small integers are converted to 4 byte integers in the file, but it is probably good point to start. You'll probably have to understand how to save the data from the usage context. Timestep string, by the way, looks ignored.

fin = open(sys.argv[1], "r")
fo  = open(sys.argv[2], "wb")

for l in fin:
    try:
        data = l.replace("\n", "").replace("\r", "").replace(" ********* TIMESTEP", "")
        if data.find(".") != -1:
            fmt = "f"
            numeric = float(data)
        elif int(data, 10)  <=127 and int(data, 10) >=-128: 
            fmt = "b"
            numeric = int(data, 10)
        else:
            fmt = "I"
            numeric = int(data, 10)
        fo.write(struct.pack(fmt, numeric))
    except:
        continue

fin.close()
fo.close()

EDIT:

Using pdf with format description (easy one) the following script translates the text to the binary almost exactly (The files are not binary identical, but sizes of the data are OK and number of differences is relatively small):

import os
import sys
import struct
import binascii


fin = open(sys.argv[1], "r")
fo  = open(sys.argv[2], "wb")

records = {
    "0": "bbf", # format
    "1": "bbfIIII", # dimensions
    "2": "bf", # timestamp
    "3": "bIIbffffffff" #vehicle
}

fmt = "UUUUU"
line_in_record = 0
for l in fin:
    data = l.replace("\n", "").replace("\r", "").replace(" ********* TIMESTEP", "")
    if line_in_record == len(fmt):
        for i in range(len(fmt)):
            if fmt[i] == 'b' or fmt[i] == 'I':
                numeric = int(read_lines[i], 10)
            elif fmt[i] == 'f':
                numeric = float(read_lines[i])
            else:
                raise Exception("Incorrect format")
            fo.write(struct.pack(fmt[i], numeric))
        line_in_record = 0
    if line_in_record == 0:
        if data in records:
            fmt = records[data]
            read_lines = []
        else:
            raise Exception("Incorrect record type")

    read_lines.append(data)
    line_in_record += 1

# writing last acquired record:
if line_in_record == len(fmt):
    for i in range(len(fmt)):
        if fmt[i] == 'b' or fmt[i] == 'I':
            numeric = int(read_lines[i], 10)
        elif fmt[i] == 'f':
            numeric = float(read_lines[i])
        else:
            raise Exception("Incorrect format")
        fo.write(struct.pack(fmt[i], numeric))

fin.close()
fo.close()

This code does not handle big/little endianess and I'd leave this problem for you to solve. As far as I understand at this point the rest of the solution is not related to reverse engineering anymore.

Good luck.

9
  • You samaritan really!! then how can i convert ascii file into binary esp in python ( i know it) and one last note could you bother to check the whole file i attached. Thanks man a lot
    – SIslam
    Dec 7 '15 at 14:32
  • Yeah! but there are strings (little) in the binary file- thanks
    – SIslam
    Dec 7 '15 at 14:46
  • Just one more thing could you tell me how to convert hex i.e. print binascii.hexlify(s) into raw byte wherefrom i generated hex. Since i may given ascii file and will be told to convert it into raw byte(i.e before hexdump)
    – SIslam
    Dec 7 '15 at 15:00
  • variable s contains the data you should write to binary file, hexlify here is just for converting it to printable form.
    – w s
    Dec 7 '15 at 15:10
  • Sir, your word Timestep string, by the way, looks ignored. have you gone through full file i attached- i need exact ascii to binary since it will be read by a software.
    – SIslam
    Dec 7 '15 at 17:57
0

hxd alpha version ( search the author's forum ) has a data inspector panel which you can manipulate to understand the format of unknown binary sequences by trial a snap shot of float value 1.04 below enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.