I am trying to find an encryption algorithm for a game client, but I need to find the actual register/stack in the client before I can decrypt/encrypt the packets correctly (on my emulated server side). The problem is that I have no idea how to find the function.

In OllyDbg, is there a way I can trace during my debugging in order to find the function right after user input? The user input is a string converted to a char array, which is then encrypted in the executable. In this case, it would mean sending a packet to the game client, and then the game client decrypts the packet.

  • What do you mean by "find the function right after user input"? On which side is the input - client-side or server-side? After that, do you mean the function that directly follows the input, a function that encrypts the input before sending the packet to the server, or a function that decrypts the packet received by the client? Please clarify in your question itself (i.e. not in a comment). Also, please give a bit more explanation on how it all works.
    – JMcAfreak
    Commented Aug 2, 2013 at 15:32
  • Do you have access to both server and client side binaries?
    – PSS
    Commented Aug 8, 2013 at 12:15

4 Answers 4


There is clearly not enough of information to get you the exact answer. However, I will try my best to demonstrate logic behind what happens. That in a way might give you an idea how to go about locating the encryption function. I will assume you are using Windows, since it has not being stated otherwise.

The main logic of ordinary (and when I say ordinary, I mean there is no deliberate attempt to obfuscate or conceal normal execution flow) server/client TCP/IP network application boils down to the following:

enter image description here

It is obvious, Send and Receive on both client and server side is where all of the exchange happening. Usually, data will get encrypted by one side (client or server). Shortly thereafter, it will get transmitted to the other side. Other side will receive the data and decrypt it:

data -> encrypt -> send <----> receive -> decrypt -> data

In order to find a function that performs encryption, you need to locate send() function, determine location of the buffer being sent, go back and see what function sets(creates/modifies) that particular buffer location.

OllyDbg comes with some features that particularly useful in this situation. Firstly, we need to find all send() and receive() functions. We will search for intermodular calls. In disassembly window right click somewhere to bring up the following menu:

enter image description here

Select Search For -> All Intermodular Calls. It will bring up the window with all of the calls found. We specifically are looking for WS2_32.recv:

enter image description here

Set breakpoint on the function by pressing F2. Run the binary. Whenever WS2_32.recv breakpoint hits you can examine the stack window and determine location of the buffer:

enter image description here

We can see, that our buffer is at location 0x001CFBB0 with the size of the buffer at 0x80, which is 128 bytes. Let's go to to the buffer location by pressing Ctrl + G and typing location. We get our buffer:

enter image description here

Now, you need to step through and see whatever happens to that buffer, and data. I would assume one of the next function will try to decrypt it.

The logic is in reverse(encrypt -> send) for sending stuff out.

Best of luck.


Your question is too broad. How is your user input taken? Try answering a few questions to narrow down the scope

SendMessage(.....WM_GETTEXT...) LB_GETTEXT 
WM_CHAR in a Loop 
scanf in a console ??
sscanf from a stram ??
Directx SendInput ??
opengl ??
touch recognition?? biometrics??  smart-card ?? voice commands

Once you narrow down to specifics you should next concentrate on narrowing down the message processing loop. Is it a classic while (TRUE) loop with switch cases for each message (Windows Message Pump) etc etc

Once the scope is narrowed down to a manageable size, several options exist to trace them. For example with:

  • Windbg you can use ( wt watch and trace when you are on a function prologue ) it will execute the function and provide you a list of subfunctions
  • with ollydbg you can use run trace feature or hit trace feature. Several plugins exist that enhance this feature
  • for ida look for Trace Replayer documentation

I hope you are on windows. If you are on another OS then GDB also has certain tracing features available.

  • the user input is just a string converted to a char array, which is then encrypted in the executable
    – bibs
    Commented Aug 2, 2013 at 15:07

If the data is ever assigned to a register then you can try OllyStepNSearch to locate it. You can find it and more info here: http://blog.didierstevens.com/programs/ollystepnsearch/


there are a lot of tutorials which could help you do that.

I would recommend you to go through Lena's reversing tutorial.

Advice: Please do not use your knowledge to crack any software. Use it constructively.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.