What are some useful strategies for tackling a list of undocumented assembly language instructions? Even as someone who writes assembly code, I find it to be challenging at times to understand someone else's code because the nature of assembly language means mentally having to keep track of CPU state at various points during the program. I've seen some success with getting a notepad and a pen and working through the program by hand. What are some other strategies? Let's take for example this function:
; cmp edx,10 ; jae internal_error push ebx ecx push eax mov ebx,eax mov ecx,edx shld edx,eax,2 sub ebx,edx sbb ecx,0 mov eax,ebx mov ebx,1999999Ah mul ebx mov eax,ecx imul eax,ebx add eax,edx pop edx imul ecx,eax,10 sub edx,ecx cmp edx,10 jb somewhere sub edx,10 inc eax pop ecx ebx retn
Due to the nature of assembly language, it is not immediately apparent "what this code actually does" at a high level. To me, the process of understanding this is true "reverse engineering" but it doesn't necessarily need to apply to reading from a disassembler, it could just be someone else's assembler source code as well. In fact, sometimes without the help of IDA Pro or Binary Ninja disassembling a binary written in a HLL, reverse engineering assembly written in aassembly is more difficult.