2

I have a number of protobuf files but no .proto schema file!

cat myfile.pbuf | protoc --decode_raw > outputfile.txt

Using the above command, I was able to decode the file into a somewhat readable protobuf format (Thanks):

1: 1
2: ""
2 {
  1: 0x40133f7ced916873
  2: 0x3ff70e5604189375
  3: 0xbfd23d70a3d70a3d
  4: 0x3fb999999999999a
}
2 {
  1: 0x4022e7ef9db22d0e
  2: 0x4006ed916872b021
  3: 0xbfe1cac083126e98
  4: 0x3fc999999999999a
}
2 {
  1: 0x402bdcac083126e9
  2: 0x40111374bc6a7efa
  3: 0xbfe9fbe76c8b4396
  4: 0x3fd3333333333333
}
2 {
  1: 0x40324147ae147ae1
  2: 0x401696872b020c4a
  3: 0xbff0e147ae147ae1
  4: 0x3fd999999999999a
}
...

I know without the schema I cannot know the meaning of these values, but I am wondering if there is anything else I can do to deduce what this strangeness is! The protobuf documentation seems to indicate that numerical data is served in 2 or 4 byte chunks, which I could easily convert to ints or floats.

My data does not fit into this format, but I know it to be numerical data! I've never seen a protobuf file with the hex x notation, and there are 16 bytes (way too many for a single number!).

What datatype might this be, is it possible to decode and further without the schema, and are the 1, 2, 3, 4 useful or significant?

  • 1
    what is this capture from? do you have access to the code which parses it? – Igor Skochinsky May 23 '17 at 21:05
  • I am using the --decode_raw command from the protoc library, which attempts to decode a protobuf stream without a .proto schema. – David Ferris May 23 '17 at 21:11
  • 1
    yes, but where is the capture from? who(what) is sending the data and who is receiving? – Igor Skochinsky May 23 '17 at 21:31
  • "I know it to be numerical data" <- how do you know it? please provide as much info as possible. – Igor Skochinsky May 23 '17 at 21:33
  • 1
    The protobuf files are intended for a piece of proprietary software, which among other things, graphs the data in a time series. The particular sample I provided corresponds to golf ball flight. The 4: seems to be a monotonic and increasing value (time)? – David Ferris May 23 '17 at 22:00
2

I suspect these are IEEE doubles. For example, 0x3fd3333333333333 is 2.99999999999999988897769753748E-1, or around 0.3. I used this converter to check

  • Is there an easy method for conversion using python? – David Ferris May 24 '17 at 15:54
  • Figure it out - you are an absolute gem!! a = struct.unpack('>d', binascii.unhexlify(line[line.index('x')+1:-1])) – David Ferris May 24 '17 at 16:07

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.