Unfortunately MSDN and windows API documentation is really scarce here, and I had difficulties finding anything other than the minimal description in MSDN.
It turns out the Vectored Continue Handlers are maintained in a Linked list very similar to the one used for Vectored Exception Handlers. They are so similar, that the function's prototypes are practically identical.
Take a look at:
PVOID WINAPI AddVectoredExceptionHandler(
_In_ ULONG FirstHandler,
_In_ PVECTORED_EXCEPTION_HANDLER VectoredHandler
);
Compared to:
PVOID WINAPI AddVectoredContinueHandler(
_In_ ULONG FirstHandler,
_In_ PVECTORED_EXCEPTION_HANDLER VectoredHandler
);
Luckily, Vectored Exception Handlers are more commonly used and documented. For example, MSDN has a page about VEHs, containing the following paragraph:
Vectored exception handlers are an extension to structured exception handling. An application can register a function to watch or handle all exceptions for the application. Vectored handlers are not frame-based, therefore, you can add a handler that will be called regardless of where you are in a call frame. Vectored handlers are called in the order that they were added, after the debugger gets a first chance notification, but before the system begins unwinding the stack.
The same page has only a laconic reference to the Add and Remove VCH APIs.
After some research and reverse engineering of ntdll, I realized VCHs and VEHs are quite similar in implementation. For example, see how AddVectoredExceptionHandler and AddVectoredContinueHandler are identical except for the VectoredListIndex, specifying they should be added to the second VectorHandlerList in the case of VCH:
Similarly, RemoveVectoredExceptionHandler and RemoveVectoredContinueHandler are identical except for the vectored handlers list index.
Inside RtlpAddVectoredHandler, the VectoredListIndex is used as an index in _LdrpVectorHandlerList, which is an array of size two of a linked list structure.
In the following picture we can see how VectoredListIndex is multiplied by the size of the list anchor object, and then added to _LdrpVectorHandlerList, which is the base offset of the array.
And now we're getting to the interesting part - where are VEH and VCH different?
If we walk up the cross references to _LdrpVectorHandlerList, we'll notice the two flows leading up to the add/remove functions are practically identical. Aside from those four APIs, we're left with only one other function, called RtlpCallVectoredHandlers which is undocumented.
It's pretty obvious from the name, but RtlpCallVectoredHandlers iterates over the vector (vector is selected according to the index) and calls all Handlers in a sequence. Once a Vectored Handler returns EXCEPTION_CONTINUE_EXECUTION the iteration is interrupted by prematurely returning from RtlpCallVectoredHandlers and execution resumes.
The sole function calling RtlpCallVectoredHandlers is RtlDispatchException, which is the main function dispatching exception handlers.
First, it executes all exception handlers, starting with the first Vectored Exception Handler to the last, and then going through all Structured Exception Handlers unfolding them through the stack. The first exception handler to return EXCEPTION_CONTINUE_EXECUTION (be it of type VEH or SEH) will stop the entire exception handlers execution process.
Like VEHs, when VCHs are called, they are called one by one until one of them returns EXCEPTION_CONTINUE_EXECUTION (just as when VEHs are called), which signals RtlpCallVectoredHandlers to break the Vectored Handlers calling loop. This is interesting because it means installing a Vectored Continue Handler as first lets you hide exceptions from subsequent VCHs.
Vectored Continue Handlers are called under the following circumstances:
- If an exception handler (either VEH or SEH) was called and returned
EXCEPTION_CONTINUE_EXECUTION
- If for some reason SEH validation failed (See SafeSEH and related mechanisms), VCHs will also be called, but the execution will not continue afterwards.
This can be seen in the many flows that lead to the second
RtlpCallVectoredHandlers call without setting bl to 1 and leaving it equal to zero before moving it to al and returning false. The calling function, KiUserExceptionDispatcher will then call ZwRaiseException if the value returned by KiUserExceptionDispatcher is false.
