I'm writing a keygen for a crack-me exercise.

I have a problem with handling byte assignments the crack-me performs several times using instructions like MOV AL,BYTE PTR DS:[ESI] (move byte from location to AL) to change EAX for example from 000096BA to 00009662.

The crack-me overflows EAX value several times so to calculate the key I use an unsigned int in my C program.

The problem I have is that I do not know how I can replace a single byte value in unsigned int example from 0x38586d to 0x38498d, changing the second byte only.

int eax;
char * esi;
*((char *) &eax) = *esi;
  • It is better to provide a thorough explanation alongside the final answer – NirIzr Mar 26 '17 at 23:04
  • &eax takes the address of eax and has the type int * (char *) &eax casts the int * to a char *. *((char *) &eax) references the char (byte) pointed to by the char *. = assigns the char / byte pointed to by esi to the first byte of eax. HTH. – Sergey Slepov Mar 26 '17 at 23:30

First, most decent compilers will let you introduce assembly directly in your C code. This is not recommended but the option should be noted.

Now, here's a sane solution; C was several bitwise operators to manipulate sets of bits inside an integer.

The bitwise operators we'll use here are:

  1. binary AND operator (&):

    A bit in the result variable is only set if it was set in both input variables. For example, 0b0011 & 0b1010 will result in 0b0010.

  2. binary OR operator (|)

    A bit in the result variable will be set if it was set in at least one of the input variables. For example, 0b0011 | 0b1010 will result in 0b1011.

  3. binary NOT operator (~)

    A bit in the result variable will only be set if it was not set in the input variable. For example, ~0b0001 will result in 0b1110.

  4. arithmetic shift operator (<<)

    A bit in the result variable will only be set if the bit n positions to the right in the first variable was set, where n is the second variable. If that position does not exist, the bit is not set. For example, 0b0000 0b0101 << 2 will result in 0b0001 0b0100.

And here's how we could use them to set the lowest byte in dword a to that of char c, assuming a 32bit processor for simplicity's sake:

unsigned int a = 0xa5a5a505;
unsigned char c = 0xa0;

First, we'll want to zero-out the lowest byte. We'll do that by ANDing the dword with a dword that has all of it's bits set except the 8 lowest bits (aka it's lowest byte).

a = a & 0xffffff00

Alternatively, we can use the NOT binary operator to create 0xffffff00 in a slightly cleaner manner, as follows:

a = a & ~0xff

After either of those lines, which perform exactly the same thing (and will look identical in assembly), a's value would be 0xa5a5a500.

Now, we'll need to assign the value of c to that same byte. We'll use the OR bitwise operator in the following manner:

a = a | c;

Which will result in a having the value of 0xa5a5a5a0.

Now, if we would like to do the same for the 2nd byte in the integer we'll shift the values by 8 bits before executing the same operators, like this:

a = a & ~(0xff << 8)

Is equivalent to:

a = a & ~(0xff00)

Which is identical to:

a = a & 0xffff00ff

Which will result with:

a = 0xa5a50005

And now, we'll add c at the 2nd byte's position:

a = a | (c << 8)

Which in our example is:

a = a | 0xa000

Which will result in 0xa5a5a005

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.