I understand that this question will be rather vague, but I'm unsure how to process from here.
I'm trying to see how a key generator can be made.
I have been toying with IDA and managed to reconstruct an application key checking functions into C up to a point that I can check if a key is valid or not. This process uses a lot of bitwise operations such as shifting and masking which leaves me wondering how I can reverse that algo.
The validation steps are as follow :
- From the serial key, use a associative table that maps chars to other values
- From those values, use a block hashing with bitshift|bitmasks so that
- Generate a key from hard coded values.
- Generate 2 tables with the result from 2. Table1 = Step2 encoded with Step3. Table2 is Table1 encoded with Step3
- Encode Step2 with tables from Step4, depending on some bitmasking.
So this whole thing is rather complicated. Furthermore there is constant loss of information so writing code with exact reverse order of operations is not possible (or I don't know how).
I put myself in the shoes of someone who wants to create this protection scheme. Generating serials from decoded output MUST be possible. Either this whole process is reversible, or there is a function that can act as a backdoor like thing to reverse it right away, which I strongly doubt.
Any idea on things that I should look for at this point?