I am trying to reverse engineer a malware that open a windows service dynamically in OllyDbg. When the malware calls StartServiceCtrDispatcherW, I receive an error: enter image description here

How I can continue to analyze this malware dynamically?

Note: I already used Image File Execution Options

  • is that StartServiceCtrlDispatcher
    – blabb
    Commented Sep 20, 2015 at 11:23
  • yes, I edit my question Commented Sep 20, 2015 at 11:56
  • 1
    More information needed. Lets being with Windows version. What exactly did you do in IFEO? Is the service an EXE or a DLL? Who runs the service EXE?...
    – conio
    Commented Nov 23, 2015 at 13:05

2 Answers 2


If you want to debug the service initialization and it happens automatically (not triggered by some action you perform), you probably can't do it on any Windows newer than XP with ollydbg. You'll have to use WinDbg.

You need to set IFEO Debugger for your process name to run CDB as server (e.g. cdb.exe -server tcp:port=12345 -noio) and the run WinDbg as a client and connect to your server (windbg.exe -remote tcp:server=localhost,port=12345).

You'll probably want to change the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ServicesPipeTimeout Registry value to be a bit longer. This is the time the SCM waits for a service it runs to talk to it.

If you don't have to debug the initialization you can simply attach to the service after it starts, and then you can use ollydbg.

All of this and pretty much everything you need to know is documented under the MSDN page titled Debugging a Service Application.

Edit: If patching the binary is possible you can add an infinite loop in the entry point like gandolf suggested and then attach a debugger after you log in.

Or, if the binary doesn't do any SEH tricks that interfere with it: Add an exception to the entry point (0xCC - int 3 is the obvious choice), set AeDebug to a long running process (such as notepad.exe) and then attach a debugger. This is what Inside Windows Debugging proposes (p. 139).

Or, what's even easier and makes even more sense: Add the same INT3, and set AeDebug to the same cdb.exe command-line as you would put in IFEO and connect WinDbg to it after you log in.

  • If the service points to an executable on disk, you can patch the service EntryPoint routine to EBFE (Inifnite jump loop) and change the ServicesPipeTimeout key, and attach to it with debugger at system startup as well. I've done this before with success.
    – gandolf
    Commented Nov 23, 2015 at 18:40
  • @gandolf, if you can edit the binary you still have options that are preferable in my opinion to adding an empty tight loop. I have edited my answer to include those options (including the loop).
    – conio
    Commented Feb 20, 2016 at 21:51

Well, as far as I know you can't debug service's handling code inside the debugger(proof).

If you really need it, I think you should stick with kernel debug of vm with malware running, but maybe someone will offer better solution - this one is the first that came to mind.


There is a link in comments that covers needed basics of kernel debugging. However, after you finish with setup of all the related things, you might face with the problem of breaking on exact place where service gets started.

I think following command will help you to go further unless you did it yourself already:

bp ADVAPI32!StartServiceCtrlDispatcherW; dd esp or use this cheat sheet for more inspiration with Windbg.

  • I don't understand your solution, can you detail your answer? Commented Sep 21, 2015 at 6:14
  • @MatanRevivo: Run your malware sample in a VM, and step through the system code with a kernel debugger attached. See Setting Up Kernel-Mode Debugging...
    – josh poley
    Commented Sep 21, 2015 at 17:02
  • 2
    There's no need to use kernel debugging except in pretty special circumstances (e.g. debugging winlogon). The problem in your so-called "proof" was that the application wasn't run as a service by the SCM, but rather directly from VS. That's what the documentation for StartServiceCtrlDispatcher says about ERROR_FAILED_SERVICE_CONTROLLER_CONNECT. That certainly doesn't mean that you can't use user-mode debugging.
    – conio
    Commented Nov 23, 2015 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.