Decipher variable length serial protocol

Question

I have an ebike consisting of a controller which regulates power to the motor and a smart LCD which can adjust max speed, power output, pedal assist sensitivity etc as well as displaying the current battery voltage, motor power usage (Watts) and motor speed.
The LCD and controller are connected by a 5v rx/tx line. I have connected an arduino mega in series in the middle of the serial lines; controller rx/tx to arduino serial1 tx/rx then arduino serial2 tx/rx to the LCD rx/tx.
Code is currently passing all received data from controller directly to the LCD and vice versa which works perfectly, LCD values are correct.

I wish to analyse the protocol but can't seem to figure out any sort of structure in it. Currently focusing on Controller -> LCD comms to extract motor values. Seems like all commands start with 02 (start of text in AsciiTable) and most then contain 0E which I initially suspected to be message length however that does not seem to be the case as evident from the following commands:

02 0E 40 40 00 00 03 0B B8 00 00 35 C8
02 0E 01 00 40 00 00 03 0B 98 00 35 C8
02 4E 40 40 00 00 03 0B B8 88 54 42 F7
02 0E 41 00 0A 00 02 03 B8 00 00 35 C8
02 0E 01 00 50 00 D0 0B B8 00 00 05 FE

02 4E 40 40 00 00 03 0B B8 00 35 C8
02 4E 40 20 0A 00 03 0A B8 80 35 C8
02 4E 40 40 00 00 03 0B B8 88 54 42 F7
02 4E 40 08 01 00 03 0B DD 81 A0 93 FE FE FE

Here is the full hexdump from roughly 60 seconds of running, messages come in constant stream: https://pastebin.com/iFFEWAFd
That dump will contain values for battery voltage ( 45-52.4V ), motor power usage (0-1000Watts, could also be 0-10Amps ) and possibly wheel speed. Likely some other params in there also.

The Command

02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8

is by far most popular accounting for over half the messages. Could this be a heartbeat possibly?
Here is the same paste without that command for easier reading: https://pastebin.com/b5iQugaY

PDF Manual for LCD: www.pedecs.co.uk
Not found any sort of manual for controller as of yet.

Thank you for any insight provided, i'm still getting my head around this whole reverse engineering thing.

UPDATE: I have managed to isolate several commands. The command previously thought to be a heartbeat is infact the speed 0. Here is a dump of the bike decelerating from 19mph to 0mph:

02 0E 01 00 40 00 00 03 00 F5 00 00 35 8E 
02 0E 01 00 40 00 00 03 00 FD 00 00 35 86 
02 0E 01 00 40 00 00 03 01 05 00 00 35 7F 
02 0E 01 00 40 00 00 03 00 0C 00 00 35 74
02 0E 01 00 40 00 00 03 01 25 00 00 35 AE 
02 0E 01 00 40 00 00 03 01 3F 00 00 35 45 
02 0E 01 00 40 00 00 03 01 4F 00 00 35 35 
02 0E 01 00 40 00 00 03 01 4F 00 00 35 35 
02 0E 01 00 40 00 00 03 01 60 00 00 35 1A 
02 0E 01 00 40 00 00 03 01 60 00 00 35 1A 
02 0E 01 00 40 00 00 03 01 76 00 00 35 0C 
02 0E 01 00 40 00 00 03 01 76 00 00 35 0C 
02 0E 01 00 40 00 00 03 01 8E 00 00 35 F4 
02 0E 01 00 40 00 00 03 01 8E 00 00 35 F4 
02 0E 01 00 40 00 00 03 01 AB 00 00 35 D1 
02 0E 01 00 40 00 00 03 01 AB 00 00 35 D1 
02 0E 01 00 40 00 00 03 01 CE 00 00 35 B4 
02 0E 01 00 40 00 00 03 01 CE 00 00 35 B4 
02 0E 01 00 40 00 00 03 01 CE 00 00 35 B4 
02 0E 01 00 40 00 00 03 01 FC 00 00 35 86
02 0E 01 00 40 00 00 03 02 8A 00 00 35 F3 
02 0E 01 00 40 00 00 03 02 8A 00 00 35 F3 
02 0E 01 00 40 00 00 03 02 8A 00 00 35 F3 
02 0E 01 00 40 00 00 03 02 8A 00 00 35 F3 
02 0E 01 00 40 00 00 03 03 0C 00 00 35 74 
02 0E 01 00 40 00 00 03 03 0C 00 00 35 74 
02 0E 01 00 40 00 00 03 03 0C 00 00 35 74 
02 0E 01 00 40 00 00 03 03 0C 00 00 35 74 
02 0E 01 00 40 00 00 03 03 0C 00 00 35 74 
02 0E 01 00 40 00 00 03 04 05 00 00 35 7A 
02 0E 01 00 40 00 00 03 04 05 00 00 35 7A 
02 0E 01 00 40 00 00 03 04 05 00 00 35 7A 
02 0E 01 00 40 00 00 03 04 05 00 00 35 7A 
02 0E 01 00 40 00 00 03 04 05 00 00 35 7A 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 06 34 00 00 35 49 
02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8 
02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8 
02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8 
02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8 
02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8

Only bytes 9,10 and 14 are changing. These values do not correspond directly to speed though, will more likely somehow relate to RPM as the LCD knows wheel size and displayed speed changes for any given command when I manually change expected wheel size value.

Update2: bytes 9 and 10 combine to output a value between 0-3000. This value is the ms between full rotations of the wheel thus allowing for calculation of speed given you know the circumference. Byte 14 appears to be a checksum of some sort, would be handy if anyone could workout how this is calculated.

Answer 1

I understand your Arduino serves as a relay in the middle between the controller and the LCD. What you could do (not extremely exciting, but you did not mention whether you made similar experiments already, and I hope I understood your situation well):

1. Modification of single bits. If you visually observe the display and connect a switch at your Arduino serving as a trigger to modify one or more single bits in the direction to the display, and observe which display indication changes.
2. The same in the opposite direction. Observe if the bike changes its behaviour.
3. Suppress your heartbeat telegrams and observe what happens.
4. An easy experiment would be to switch on the bike's light and observe the changes in the telegrams.
5. Add timestamps to your telegrams. If you install a trigger at your Arduino, give it a timestamp too, to have the exact change time in the telegram log.

In short, try to make changes (one at a time) in a defined way either at the bike or in the telegrams or groups of telegrams and observe if it gives you hints about the content.

Another question would be if you cross-checked your Arduino telegram decoding, e.g. with some passive sniffer. I understand you trust your decoder because everything works as before (without the Arduino in between). But this might simply be due to the fact that corrupted telegrams are discarded.

A coarse inspection of your log shows properties not easily understood.

Example: The lines #133 and #134 in the file ebike_sw_hex_dump.txt (the one with the heartbeat):

02 0E 01 00 40 00 00 03 0B B8 00 00 35 C8
02 0E 01 00 40 00 03 0B B8 00 00 35 C8

The second telegram is identical to the first one except from the missing zero in the middle. In particular the last seven bytes are identical. If I assume that the logical content of these last seven bytes is identical in both telegrams, how should an interpreter come to this conclusion?

The "protocol switch" would probably be the byte after the 40. This however would contradict the assumption that the last seven bytes mean the same.

From this example and other strange lines, e.g. those with the FE or FF at the end, I would suggest an independent cross-check of the decoder, just to make sure that your log shows the correct bytes. Similar as you suspected, in such a system I would have expected constant telegram lengths as well.

Of course, there is more which could be tried. These were just some simple ideas which came into mind.

A similar experiment as yours is published in the web, under https://endless-sphere.com/forums/viewtopic.php?f=2&t=73471#p1109048, decoded protocol attached, however with a different controller, and showing no obvious similarity to your logs. But maybe it gives some hints as well.

In any case, a nice study!