How to debug packet processing with overlapped socket on Windows

Question

The problem is quite complex so i don't want to run into unneccessary details.

I have a network application, that communicates with multiple clients at once via UDP. I haven't written it and there is no way to get source codes. Its author has abandoned it so i cannot expect any kind of support from his side. That server application contains a bug, that allows anyone to crash it by sending a specificaly malformed packet. I need to develop some kind of fix on my own.

I'm trying to analyze the program using IDA Free, x64dbg and some other tools. However i am unable to find the code, which is processing the received packets. First i've looked up the API calls and found a call to WSARecvFrom (documentation by MSDN) and set a breakpoint to it. But inspecting its arguments i found out, it is using overlapped sockets. I learned little bit about this model here. According to that article, the receive operation is performed asynchronously and completed later and there are 2 ways how the program can be notified when it's finished.

1. by a pointer to a function lpCompletionRoutine, which gets called when data are ready
2. by a HANDLE to an event object, that is set, when data are ready

However in my application both lpCompletionRoutine and lpOverlapped.hEvent are NULL. Yet the function always return -1 and WSAGetLastError() is 3E5 (WSA_IO_PENDING).

If lpCompletionRoutine and lpOverlapped.hEvent are both NULL, how does the application know, when to process the received data and more importantly, how do i find that code, which processes those data??

EDIT: I have tried searching API call WSAGetOverlappedResult, there isn't any. I also tried setting a hardware breakpoint on that buffer, but it didn't break until exception ACCESS_VIOLATION, where data were already in buffer and being processed.

I need to find the beginning of the processing, but there is quite a lot of code between.

Answer 1

So the problem was, the network application was using GetQueuedCompletionStatus instead of WSAGetOverlappedResult. It is explained here: https://support.microsoft.com/cs-cz/help/181611/socket-overlapped-i-o-versus-blocking-nonblocking-mode. I just somehow missed this article.

If you're asking how i found it out, well i had to do it the hard way. I started at the instruction causing ACCESS_VIOLATION, and i went backwards in the code flow using return addresses on stack trying to trace the origin the malformed data. Unfortunately the crash was happening in other thread than packet processing, so i also had to use hardware memory breakpoints to find, where are those data copied from, etc.

One mistery however remains unsolved, and that's: Why didn't a hardware breakpoint put directly in the buffer passed to WSARecvFrom pause my program? When code reaches GetQueuedCompletionStatus, data are clearly there, but it passes my breakpoint and performed this operation without breaking.