The key is getting getting a megaton of samples, so that the analysis has something to feed on. It really helps if you can stick the samples in a database table or dictionary that can be queried interactively, e.g. from some sort of script shell. Python should work admirably but I don't have much experience with it, as I've been using Visual FoxPro for interactive spelunking these last two decades.
Once you have your samples ready to be queried you can test various simple hypotheses and disprove them by finding counterexamples. For example, the following samples seem to suggest that a difference in the fourth byte causes a quadruple difference in the checksum:
00h 5Ch A2h 00h 00h 01h 7Fh
00h 5Ch A2h 01h 00h 01h 63h
00h 5Ch A2h 02h 00h 01h 67h
00h 5Ch A2h 03h 00h 01h 6Bh
00h 5Ch A2h 04h 00h 01h 6Fh
The key is finding samples that differ only in the current 'working' column and the checksum, to study the effect which a change in the working column has on the checksum. For example, given the input above we could note down 4 * b[3]
as preliminary term for the fourth byte, and then query the sample database for counterexamples were this doesn't work. That is to say, you select pairs of samples that differ only in the fourth byte and the checksum and count the number of successes and failures for
delta(checksum) mod 256 == 4 * delta(byte[3]) mod 256.
That's the idea, anyway. What a thing like delta(checksum) mod 256
actually looks like depends on your script shell, obviously. With VFP it would be
mod(256 + asc(substr(right, 7, 1)) - asc(substr(left, 7, 1)), 256)
The complete expression for the test above would be rather long, so you'd normally write little helper functions. With a helper named byte_delta()
that normalises its result to the range [0, 255] you might have
select le.sample as left, ri.sample as right ;
from all_samples le, all_samples ri ;
where stuff(left(ri.sample, 6), 4, 1, "") == stuff(left(le.sample, 6), 4, 1, "") ;
and le.sample <> ri.sample ;
into cursor byte_3_pairs
select byte_delta(right, left, 7) == mod(4 * byte_delta(right, left, 4), 256) as ok, count(*) ;
from byte_3_pairs ;
group by 1
In the case under consideration you'd get a mixed picture for this query (see difference between first and second sample above, which is E4 instead of 4). Normally you'd do a fair bit of selecting differences first, just to get a feel for things. 'Difference' could be arithmetic difference, bitwise xor, whatever.
That's why you need an interactive shell like Python or VFP; with the edit-compile-run cycles of a compiled language this would be rather cumbersome. I've given the VFP examples because scripting languages with database support can make things a lot easier here.
For simple weighted sums like the usual 'human-computable' checksums in things like ISBNs this gives fairly quick results. I've used this approach to determine all the check digit schemes used in German health insurance numbers - which are mostly undocumented (or at least were, at the time) - from varying amounts of samples. Of course, in that case I had the advantage that the basic type of scheme - a weighted sum of digits - was known, and the samples were already residing in database tables...
The case under discussion is more difficult because the basic scheme is not yet known. That's why it is important to look at the bit patterns to get a feel for things. For example, bubbling carries indicate an additive function. This is explained in more detail in the topic Reversing simple message + checksum pairs (32 bytes), which also shows a few change pattern examples.
P.S.: besides trying to tease out checksum differences for changes in specific bits or bytes, there are lots of other things that can be done when the samples have been stuffed into some sort of queryable table/dictionary/map. The first thing is usually to run a battery of tests with existing standard functions, like straight byte sum, straight byte xor, various CRCs and so on, to observe the difference (arithmetic and xor) to the checksum. Displaying results as bit patterns - as shown in the linked article - can often be helpful for discerning regularities that are not so obvious in hexadecimal or decimal format.
UPDATE At the moment the samples are too similar (no differences in the first three bytes) and there's too few of them for discarding hypotheses quickly. In other words, there are just too many potential functions that would fit the existing data...
For example, the following simple Fox function correctly predicts the checksum for the original handful of samples, except for a few cases where it is off by 0x20:
function f (s)
local x, i
x = 0
for i = 1 to len(m.s) - 1
x = bitand(bitlshift(m.x, 1) + asc(substr(m.s, m.i, 1)), 0xFF)
next i
return bitand(m.x + 0x8E, 0xFF)
That could be because it's a rotation instead of a shift, or some xor is involved, or some other shift (relatively prime to 8) that overlaps with the bitcount per byte to create a distance of two bits between the fourth byte and the checksum. A rotation by 5 would do that. However, there are a lot of other possibilities... That's why we need lots more samples. ;-)
Analysing the samples on PasteBin shows that a difference in the last byte position before the checksum is always equal to the difference in the checksum. This means that the last byte and its effect on the checksum can be removed from the sample base. This increases the number of samples that differ in just one byte position, which means there's more low-hanging fruit for analysis...
E.g. the samples
00 5C A0 00 00 00 00 00 06 00:69
00 5C A0 00 00 00 00 00 06 01:6A
...
00 5C A0 00 00 00 00 00 06 FF:68
all map to the new sample (the dot indicates a removed byte, just for the sake of exposition here):
00 5C A0 00 00 00 00 00 06 . 69
The shortened sample base immediately shows cases where the 'power of two' rule does not work (the last byte here is originally the second last):
00 5C A0 00 00 00 00 00 00 . 7D
00 5C A0 00 00 00 00 00 01 . 7F
00 5C A0 00 00 00 00 00 02 . 61 <- difference -0x20 to predicted delta 2
00 5C A0 00 00 00 00 00 03 . 63
00 5C A0 00 00 00 00 00 04 . 65
...
00 5C A0 00 00 00 07 00 00 . B5
00 5C A0 00 00 00 08 00 00 . BD
00 5C A0 00 00 00 09 00 00 . A5 <- difference -0x20 to predicted delta 8
00 5C A0 00 00 00 0A 00 00 . AD
00 5C A0 00 00 00 0B 00 00 . D5 <- back on track with the earlier sequence
0x7e + d[5] + 2*d[4] + 4*d[3] - (d[3] + d[4] == 0 ? 0 : 0x20)
(d[0..5] being the data). If you can get more samples like the failing one, or samples that vary the first three bytes, you can probably refine it further.