The problem is quite complex so i don't want to run into unneccessary details.

I have a network application, that communicates with multiple clients at once via UDP. I haven't written it and there is no way to get source codes. Its author has abandoned it so i cannot expect any kind of support from his side. That server application contains a bug, that allows anyone to crash it by sending a specificaly malformed packet. I need to develop some kind of fix on my own.

I'm trying to analyze the program using IDA Free, x64dbg and some other tools. However i am unable to find the code, which is processing the received packets. First i've looked up the API calls and found a call to WSARecvFrom (documentation by MSDN) and set a breakpoint to it. But inspecting its arguments i found out, it is using overlapped sockets. I learned little bit about this model here. According to that article, the receive operation is performed asynchronously and completed later and there are 2 ways how the program can be notified when it's finished.

  1. by a pointer to a function lpCompletionRoutine, which gets called when data are ready
  2. by a HANDLE to an event object, that is set, when data are ready

However in my application both lpCompletionRoutine and lpOverlapped.hEvent are NULL. Yet the function always return -1 and WSAGetLastError() is 3E5 (WSA_IO_PENDING). WSARecvFrom call

If lpCompletionRoutine and lpOverlapped.hEvent are both NULL, how does the application know, when to process the received data and more importantly, how do i find that code, which processes those data??

EDIT: I have tried searching API call WSAGetOverlappedResult, there isn't any. I also tried setting a hardware breakpoint on that buffer, but it didn't break until exception ACCESS_VIOLATION, where data were already in buffer and being processed.

I need to find the beginning of the processing, but there is quite a lot of code between.

  • This is really more of a programming question, but are there calls to WSAGetOverlappedResult?
    – Igor Skochinsky
    Feb 12 '17 at 21:37
  • Unfortunately, there aren't, i have already searched for them, should have written it in the question. I also tried hardware breakpoint on that buffer, but it didn't work and i don't understand why. Look at the edited question.
    – Youda008
    Feb 13 '17 at 14:49
  • So the problem was, it was using GetQueuedCompletionStatus instead of WSAGetOverlappedResult. I had to find it the hard way - going back in code flow through return addresses on stack and setting hardware memory breakpoints on various places to trace where those data came from.
    – Youda008
    Feb 15 '17 at 17:01
  • maybe you can write up a detailed answer on how exactly you did it?
    – Igor Skochinsky
    Feb 15 '17 at 17:03

So the problem was, the network application was using GetQueuedCompletionStatus instead of WSAGetOverlappedResult. It is explained here: https://support.microsoft.com/cs-cz/help/181611/socket-overlapped-i-o-versus-blocking-nonblocking-mode. I just somehow missed this article.

If you're asking how i found it out, well i had to do it the hard way. I started at the instruction causing ACCESS_VIOLATION, and i went backwards in the code flow using return addresses on stack trying to trace the origin the malformed data. Unfortunately the crash was happening in other thread than packet processing, so i also had to use hardware memory breakpoints to find, where are those data copied from, etc.

One mistery however remains unsolved, and that's: Why didn't a hardware breakpoint put directly in the buffer passed to WSARecvFrom pause my program? When code reaches GetQueuedCompletionStatus, data are clearly there, but it passes my breakpoint and performed this operation without breaking.

  • 2
    Re the last paragraph: hw breakpoints set in user mode won't catch data being written from the kernel mode. You need to use a kernel debugger for that.
    – Igor Skochinsky
    Feb 18 '17 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.