Skip to main content
14 of 23
added 1 character in body
Simon
  • 31
  • 2

Simplifying and reversing an encryption/decryption algorythn in ASM X86 / Python

I'm trying to reverse a decoding function in order to produce the function for encoding.

So far I have ported the decoding function from ASM X86 to Python and it's working fine. My problem is I'm not able to reverse the function because I don't understand what the code does with the bitwise operations. Specially the bitwise OR is like voodoo to me.

I'd like some help to simplify/understand the function and reverse it.

My code so far:

def text_to_hex(text):
  with open("DECODE_TABLE.bin", mode='rb') as file:
    DECODE_TABLE = file.read()

  decoded = []
  char_index = 0
  HARDCODED_LIMIT = 38
  i = 0

  while i < HARDCODED_LIMIT:
    if char_index >= len(text):
      break
    current_char_1 = ord(text[char_index])
    char_index += 1
    char_0 = DECODE_TABLE[current_char_1]
    if char_0 == 0xFF:
      break
    if char_index >= len(text):
      break
    current_char_2 = ord(text[char_index])
    char_index += 1
    char_1 = DECODE_TABLE[current_char_2]
    if char_1 == 0xFF:
      break
    char_next_1 = ((char_0 & 0xFF) * 2048) | ((char_1 & 0xFF) * 64)
    char_1 = char_next_1 >> 8
    if i >= HARDCODED_LIMIT:
      break
    decoded.append(char_1)
    i += 1


    if char_index >= len(text):
      break
    current_char_3 = ord(text[char_index])
    char_index += 1
    current_char_3 &= 0xFF
    char_2 = DECODE_TABLE[current_char_3]
    if char_2 == 0xFF:
      break
    if char_index >= len(text):
      break
    current_char_4 = ord(text[char_index])
    char_index += 1
    char_3 = DECODE_TABLE[current_char_4]
    if char_3 == 0xFF:
      break
    char_next_2 = ((((char_2 & 0xFF) * 2) | char_next_1) * 16777216) | ((char_3 & 0xFF) * 1048576)
    char_3 = (char_next_2 >> 0x18) & 0xFF
    if i >= HARDCODED_LIMIT:
      break
    decoded.append(char_3)
    i += 1


    if char_index >= len(text):
      break
    current_char_5 = ord(text[char_index])
    char_index += 1
    char_4 = DECODE_TABLE[current_char_5]
    if char_4 == 0xFF:
      break
    char_next_3 = char_next_2 | ((char_4 & 0xFF) * 32768)
    char_4 = (char_next_3 >> 0x10) & 0xFF
    if i >= HARDCODED_LIMIT:
      break
    decoded.append(char_4)
    i += 1


    if char_index >= len(text):
      break
    current_char_6 = ord(text[char_index])
    char_index += 1
    char_5 = DECODE_TABLE[current_char_6]
    char_next_3 = char_next_3 | ((char_5 & 0xFF) * 1024)
    if char_index >= len(text):
      break
    current_char_7 = ord(text[char_index])
    char_index += 1
    char_6 = DECODE_TABLE[current_char_7]
    if char_6 == 0xFF:
      break
    char_next_4 = char_next_3 | (char_6 * 32)
    char_6 = (char_next_4 >> 8) & 0xFF
    if i >= HARDCODED_LIMIT:
      break
    decoded.append(char_6)
    i += 1


    if char_index >= len(text):
      break
    current_char_8 = ord(text[char_index])
    char_index += 1
    char_7 = DECODE_TABLE[current_char_8]
    if char_7 == 0xFF:
      break
    char_8 = ((char_next_4 & 0xFF) | (char_7 & 0xFF)) 
    if i >= HARDCODED_LIMIT:
      break
    decoded.append(char_8)
    i += 1

  return decoded

The content of the DECODE_TABLE.bin file:

FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF 00 01 02 03 04 05 06 07 08 09 FF FF FF FF FF FF FF 0A 0B 0C 0D 
0E 0F 10 11 01 12 13 01 14 15 00 16 17 18 19 1A FF 1B 1C 1D 1E 1F FF 
FF FF FF FF FF 0A 0B 0C 0D 0E 0F 10 11 01 12 13 01 14 15 00 16 17 18 
19 1A 1B FF 1C 1D 1E 1F FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FF FF FF                                                             
Simon
  • 31
  • 2