What is all this code for?

Question

In order to be able to effectively reverse assembly one needs to understand how different compilers work, and what are the common code generation patterns. If you do not know them, you spent much more time figuring out something that is simply an artifact of compiler code generation.

Given the above, I'd like to understand meaning of the following code. It's from lzham_codec library but as you will see in a moment it's not specific to it. I clone this git repo and then I compile the code with VS2015. I set a breakpoint to lzham_codec\lzhamdll\lzham_api.cpp (line 9)

Then I run lzhamtest and the breakpoint hits. I open the Disassembly window, and here is what I see:

This is an awful lot of code for a function that simply returns a constant. What is all this code for?

Answer 1

I'll assume you have a general understanding of the x86 processor architecture, registers, and how the stack works. If you don't, there are a lot of introductions, tutorials, and books out there, which explain things much better than I could in this post.

The first 2 and last 3 instructions are standard function entry/exit code:

push ebp
mov ebp, esp
....
mov esp, ebp
pop ebp
ret

they set up the stack for the function, resp. undo this and return to the caller.

sub esp, 0C0h

makes space for 192 (0xC0) bytes of local variables on the stack.

The push and pop instructions save registers to the stack that get clobbered by the function, and restore them afterwards:

push ebx
push esi
push edi
...
pop edi
pop esi
pop ebx

Note that ebx and esi aren't even touched by the rest of the code. But it seems you compiled without optimizations; the abi states that these registers shouldn't be changed by procedures, so the compiler saves them, and without optimizations, it doesn't realize it doesn't need them later and doesn't remove the push? /pop` from the code.

lea edi, [ebp-0C0h]
mov ecx, 30h
mov eax, 0cccccccch
rep stos dword ptr es:[edi]

This fills all local variables with 0xCCCCCCCC, but your C code shows no reason for that. Maybe it has to do with some variable declarations that isn't shown in the code, or maybe the compiler just initializes local variables to 0xCCCCCCCC to prevent "uninitialized variables have undefined values, don't let the code assume they are zero" errors.

What remains is

mov eax,1010h

which is the return instruction - function results are generally returned in the eax register, and 0x1010 seems to be what LZHAM_DLL_VERSION is defined to.