The function commented as "New Exception Handler", does contain a Cookie check. But I've never been able to step into it.
Since the exception handler is called only during an exception, it won't be called during normal execution, so this is expected behavior.
As for the missing cookie check in the epilog, apparently your function is not using the plain old
GS cookie (return address overwrite protection via simple comparison) but only the improved
EH+GS cookie (local variables + EH data + return address protection with the extra XOR check).
Commented function prolog (comments refer to final offsets in the frame):
push ebp ; [EBP+0] save old EBP
mov ebp, esp ; set up ebp frame (ESP=EBP)
push 0FFFFFFFFh ; [EBP-4] push initial state
push offset SEH_1000BF50 ; [EBP-8] SEH handler for the function
mov eax, large fs:0 ; read SEH chain list head
push eax ; [EBP-C] pointer to previous SEH record
sub esp, 14h ; allocate space for local vars and move to the saved registers area.
push ebx ; [EBP-18] save ebx
push esi ; [EBP-1C] save esi
push edi ; [EBP-20] save edi
mov eax, ___security_cookie
xor eax, ebp ; xor the cookie to make it harder to forge
push eax ; EBP-24 push XOR'ed cookie
lea eax, [ebp-0Ch] ; eax = &SEH_record
mov large fs:0, eax ; insert our SEH entry at start of list
final stack frame layout:
EBP-30 xored EH cookie
EBP-2C saved edi
EBP-28 saved esi
EBP-24 saved ebx
EBP-20 <local variables>
EBP-0C SEH pointer to previous record\
EBP-08 SEH handler | extended SEH record
EBP-04 EH state /
EBP+00 saved EBP
EBP+04 return address
In case of an exception, the check happens in the function-specific SEH handler before jumping to
___CxxFrameHandler3 which performs C++ exception handling or stack unwinding:
mov edx, [esp+8] ; (1) get pointer to current SEH record
lea eax, [edx+0Ch] ; edx <- original frame pointer (function's EBP)
mov ecx, [edx-24h] ; get xor'ed cookie at orig_ebp-30h
xor ecx, eax ; xor them together
call @__security_check_cookie@4 ; check expected value
mov eax, offset stru_10034344
jmp ___CxxFrameHandler3 ; handle C++ exceptions/unwinding
(1) gets the second argument of the SEH handler (
EstablisherFrame), which points to the active SEH record which was stored at
EBP-C in the function, so we adjust it by
0Ch to get the original
EBP value which is used for xoring with the cookie value (originally stored at
[EBP-30h], or at -24h from the SEH record), and the xor result is compared with the expected value by
I guess the compiler decided that there is no need to protect just the return address because there are no buffers in the function (see "GS Buffers" in MSDN), and any other overwrite will trash the EH cookie and will be detected in case of exception.
state variable at
[ebp-4] uniquely identifies a state in the function's execution. This allows the C++ exception handler (e.g.
__CxxFrameHandler) to destroy any automatic objects leaving their scope due to the exception and unwind the state to the starting one (usually value -1). So this exception handler setup does not correspond to a "try/catch block" but has been added by the compiler for the whole function. However, the state transitions (writes to the try level variable may correspond to try block boundaries (or just scope boundaries).
For more background on Visual C++ exception handling implementation (both C-style SEH and C++ EH), check out my article on the topic (and its references).