1

I want to write a C function, such that hex-rays decompiler will fail on it. I want to do it for study purposes, and not for an actual anti-reversing method. Do you have any recommendations/approaches how can I write such a function, that will compile with gcc or similar compiler, but won't be decompilable by hex-rays?

EDIT: My goal is to make the code disassemblable, but not decompilable. I'm not looking for obfuscators that will hide the code completely, but a way to make IDA not to be able to decompile. for example, by somehow messing with the stack pointer.

8
  • 2
    There's a concept called obfuscation. That's what you're looking for. Try giving the tag by the name name a peek: reverseengineering.stackexchange.com/questions/tagged/…
    – NirIzr
    Commented Dec 26, 2019 at 13:48
  • Thanks, my question wasn't very clear, but I edited it now. Anyways I will go through your link and maybe find something relevant. Commented Dec 26, 2019 at 15:05
  • why are you be interested specifically in preventing decompilation and not disassembly? for the record, though, obfuscated code is disassembleable, just not to anything too intelligible.
    – NirIzr
    Commented Dec 26, 2019 at 15:13
  • Does the example in my question count? Or you want another example? I observe that IDA does not care much in detecting function types. Commented Dec 26, 2019 at 15:31
  • @NirIzr I think much effort have focused on anti-disassembly. While it's correct that anti-disassembly will help anti-decompilation, they are not the same. Commented Dec 26, 2019 at 15:38

2 Answers 2

1

You can check "graceful failures" for some common problems that can prevent decompilation and try to induce them deliberately. However, most of them can be worked around with a bit of effort so don’t expect them to stop a motivated person.

0

The answer of @igor says about algorithm failures of IDA. Beside, I think you can use a function which is not type-able in the type system of the decompilation target language of IDA (I believe it's a subset of C), e.g.

int foo(void *f, int i) {
    return ((int (*)(void*, int))(f))(foo, i);
}

clang -c test.c

then IDA decompilers gives something likes

__int64 __fastcall foo(__int64 (__fastcall *a1)(__int64 (__fastcall *)(), _QWORD), unsigned int a2)
{
  return a1(foo, a2);
}

which is obviously not correct: foo is recognized simultaneously as a function of two params and one param.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.