I'm in the process of decoding the semantics of an unknown binary stream1. I've come to a point where I understand the overall structure: A tag
byte, followed by a size field, and a blob of bytes of that size, e.g. 80 02 42 0a
(type: 0x80, size: 2 bytes, data: 66, 10).
To help me drill deeper I'm looking for a tool that allows me to extract the individual chunks of data, run queries against those, and arbitrarily transform the results. I'm looking for simple queries (like "all packets with a specific tag"), but also need more complex filters (e.g. "distribution of all size values for packets with a specific tag").
Are there any tools that allow me to easily perform those queries, given the following requirement?
- Needs to understand the overall stream structure.
- Needs to have basic decoding capabilities (e.g. to extract base128-encoded
varint
s as used by Protocol Buffers). - Needs customizable filters (e.g. by tag, size, or data).
- [Optional] Ability to control display of data, like 64-bit FILETIME values.
- [Optional] Ability to render graphs.
- Needs to run on Windows (preferably free).
1 The data is streamed off of an embedded device. I have access to fully self-contained chunks of that data as binary files.
tag
size
<blob of data>
. I don't understand<blob of data>
yet, and I need help with that. To get there, I need to test hypotheses, by collecting statistical data. – IInspectable Jan 17 '20 at 20:56