3

I am trying to figure out where the code is that interacts with the Serial Port on an MFC Win32 binary - and am utterly out of ideas on how to find this code block.

I have even tried, say, stopping the process right before I click the button to initialize serial communication and then stepping through. The program returns with an error immediately.

I have even tried setting a breakpoint over the string text of the error message in the .data segment, and even then no breakpoint.

What is causing this? Why can't I just step through a process to see where its contacting the serial port?

How is it possible that its hiding at this level?

Improve this question
1
  • What tools do you have at your disposal to do the job? Also, depending on the program it could use/require a specific driver through which it talks to the serial port or it could use simply CreateFile or its native counterpart NtCreateFile. But in general when communication with a driver is involved you can expect there to be a call to something like CreateFile. Windows is not so dissimilar to Unix, since the object manager namespace also has just a single root and you talk to devices the same way as you read/write files.
    – 0xC0000022L ♦
    Oct 21, 2016 at 8:38

2 Answers 2

Reset to default
1

I ended up using the tool API Monitor. It took a ton of filtering, but I eventually found a reference to CreateFileA. Odd, because Ida Pro does not reference CreateFileA anywhere. Either way, it works! Thank you!

Improve this answer
2
  • 1
    Is it a statically or dynamically linked executable? If dynamic, the call to CreateFileA will be probably be in one of the MFC dlls, not in the executable itself.
    – Ian Cook
    Oct 22, 2016 at 6:11
  • MFC apps are more likely to use something like CFile or CArchive class for I/O than direct API calls.
    – Igor Skochinsky ♦
    Oct 22, 2016 at 7:33
1

You just have to attach to the desired process with ollydbg or something similar and breakpoint ReadFile,WriteFile and CreateFile. (If you also need the information about the connection itself like baudrate or stuff like that you will have to set a breakpoint at SetCommState too.)

After hitting a breakpoint you just have to follow the pointer on top of the stack to get to the position the current function got called.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.