Use OllyDbg to identify what happens when a button is pressed

Question

I am entirely new to reverse engineering, but nonetheless I have been given the challenge by a friend to crack a computer program he wrote. Having never done this before, my research online suggested that I use OllyDbg to search for an error message given when I try to give an invalid code. The problem is that if the code is invalid, the button to submit it is never enabled. Could I use OllyDbg to find what would happen if I were to click the button and then somehow override it?

Here's all I know about the program:

• I do not have access to the source code, nor do I know what language it was written in.
• The program can be activated as was proven to me by my friend has he typed in a key to activate it, yet covering it so I could not see it.

Given that I have never dabbled in reverse engineering before, what would be my best plan of attack and how would I go about it?

Answer 1

If you have very little experience in binary RE I would suggest to start with preparing for a lot of unknown information that would be "thrown" on you, time and patience :-).

Now to the subject. To do the work your are talking about, you'll need tools and you need to know them well:

• Olly - to do the dynamic analysis of the binary. The one you already have and I hope opened it already to get now the tool.
• IDA - to do the static analysis. It will help you navigate the binary. You can download from the supplied link the free version which will be sufficient for your task.
• CFF explorer - to understand the type of the binary and poke around with its dependencies, resources.

Next, supposedly the binary is not obfuscated:

• Try to execute the binary and study a little bit its behaviour and take notes - strings (titles, messages), button names. Give it some input and look for the results.
• Based on binary type:
  • .Net
    • check with ILSpy to get the sources if possible.
  • Native
    • Load it into IDA and check if you can find/verify from notes the data inside the binary.
    • As this is the GUI application, I would check with MSDN in the area of GetMessage and TranslateMessage, CreateWindow and look for Window proc which is responsible for processing window messages.

After the above, I would also suggest, as @Dominik did, go over the fist Lena's tutorials, to cover all the needed info for your type of binary.

I think the above can pretty much get you to the key you are looking for. Add more info from the ongoing analysis and we will try to provide another help.

Good luck!

Answer 2

I assume your friend did not utilize tricks like anti-debugging and packing the binary, as your first challenge. Assuming is not right for an answer, so you should provide more details next time :) How I would approach this:

1. Go trough Lena's tutorials.

2. Identify what is the application was written in.

   There are multiple tools for this, such as :

   • PEID or ProtectionID.
   • More can be found here. Play with them :)

3. So, from the sounds of it your friend has something like this in place:

   void onEachKeyPress() // meaning each time you type a character into
                         // the text box, this function will be called
   {
      if(txtBoxInput == "secretKey123#")
      {
        EnableButton();
        MessageBox("You did it!");
      }
   }
   

   So, we know that the button will be activated if the textbox contains the right key. Also there is probably a gratulation message after entering the right key and pressing the button. From this you can assume that there will be a function for enabling the button, or s a MessageBox window.

   Eventually there will be a condition somewhere in the application, which decides if you have the right key or not, if there is no trickery involved the key itself will be in the application somewhere.

   Olly will show you machine language, and conditions (if) will translate to jump operands (je, jz, jne, ...).

   

   (the action in this case would be enabling the button and showing a message)

4. So now you know all these, it is time to look round. Run the app, attach Olly, or you can start your app from olly, and set it to break on the main function. This can be handy, because from there you can step by step follow the execution flow of the application. You now know what functions to look for. Put some breakpoints and observe. List all strings in the binary. See if any looks like a hardcoded password.

   If you find the piece of code that decides if your input is right or not, you could patch it, or just simply find the hardcoded password and enter it without modifying (patching) the binary.

   If you run out of things to crack, and want to move on to more advanced challenges, head over to crackmes.de.

Answer 3

The explanation below goes only for native apps: If you want to understand how to find out where buttons' events occur, I advice you to learn coding in these languages: ASM, C++, and Delphi (at least get a general approach about them). Then, try to find where the event occurs of your code in the compiled file.

Use google translation for this page, I wrote about this stuff once: http://www.at4re.com/f/showthread.php?11276

Answer 4

Set a breakpoint on the beginning of that function, hit that breakpoint, and step through to see what happens. That's the most cut and dry answer there is really.