Skip to main content
added 1387 characters in body
Source Link
Tobias
  • 201
  • 1
  • 4

There are several things in your example that makes it hard to decompile.

s is the first, and only, local (so on the stack) variable in main(). main() is troublesome, as it's more or less a vararg-function if you read the C++ standard, and as you can see atleast IDA guesses that you have three arguments on the stack.

You use both int and long in your struct definition, which may or may not create either padding of the stack or masking in the generated code. It can also be one way where you declare it (main) and another way when passing it by value to a (leaf-)function.

And, foo() is a leaf-function, meaning it will have a red-zone on the stack that could possibly be used.

Try putting s on the heap instead, and you will probably see a very different result :)

What does the disassembly look like?

Edit: Oh the disassembly really drives the point home! The point being that LLVM depends on how well-suited the IR is to optimization, as before optimization the code looks like someone who licks rocks built it from lego. And then threw the rock at it :D No wonder it confuses decompilers :) Look at that funny byte-size "bonus parameter" and the "nonsensical" movsx-instructions for example.

Anyways, serious face-time again. The red zone isn't used. The prologue isn't even needed as nothing is stored on the stack, all calculations are done on RCX and RAX. Now that you've gotten rid of any stack-variables in main(), the thing tripping you up is that you're passing a small, stack-allocated, structure by value. What in C looks like passing a single blob as argument is actually treating each field like a separate argument. I am guessing both IDA and Ghidra would be able to make sense of this if it wasn't for the "alignment"(?)-byte thrown in there. Or perhaps not, as the assembly might still look like it's passing four separate arguments on the stack :|

Tl;dr: clang generates really strange code unless optimized. Coupled with passing a stack-allocated struct by value it will confuse the hell out of both decompilers and sleepy reverse engineers such as myself. Take this opportunity to kick the habit of passing structs by-value and learn to love the const-refs ;)

There are several things in your example that makes it hard to decompile.

s is the first, and only, local (so on the stack) variable in main(). main() is troublesome, as it's more or less a vararg-function if you read the C++ standard, and as you can see atleast IDA guesses that you have three arguments on the stack.

You use both int and long in your struct definition, which may or may not create either padding of the stack or masking in the generated code. It can also be one way where you declare it (main) and another way when passing it by value to a (leaf-)function.

And, foo() is a leaf-function, meaning it will have a red-zone on the stack that could possibly be used.

Try putting s on the heap instead, and you will probably see a very different result :)

What does the disassembly look like?

There are several things in your example that makes it hard to decompile.

s is the first, and only, local (so on the stack) variable in main(). main() is troublesome, as it's more or less a vararg-function if you read the C++ standard, and as you can see atleast IDA guesses that you have three arguments on the stack.

You use both int and long in your struct definition, which may or may not create either padding of the stack or masking in the generated code. It can also be one way where you declare it (main) and another way when passing it by value to a (leaf-)function.

And, foo() is a leaf-function, meaning it will have a red-zone on the stack that could possibly be used.

Try putting s on the heap instead, and you will probably see a very different result :)

What does the disassembly look like?

Edit: Oh the disassembly really drives the point home! The point being that LLVM depends on how well-suited the IR is to optimization, as before optimization the code looks like someone who licks rocks built it from lego. And then threw the rock at it :D No wonder it confuses decompilers :) Look at that funny byte-size "bonus parameter" and the "nonsensical" movsx-instructions for example.

Anyways, serious face-time again. The red zone isn't used. The prologue isn't even needed as nothing is stored on the stack, all calculations are done on RCX and RAX. Now that you've gotten rid of any stack-variables in main(), the thing tripping you up is that you're passing a small, stack-allocated, structure by value. What in C looks like passing a single blob as argument is actually treating each field like a separate argument. I am guessing both IDA and Ghidra would be able to make sense of this if it wasn't for the "alignment"(?)-byte thrown in there. Or perhaps not, as the assembly might still look like it's passing four separate arguments on the stack :|

Tl;dr: clang generates really strange code unless optimized. Coupled with passing a stack-allocated struct by value it will confuse the hell out of both decompilers and sleepy reverse engineers such as myself. Take this opportunity to kick the habit of passing structs by-value and learn to love the const-refs ;)

Source Link
Tobias
  • 201
  • 1
  • 4

There are several things in your example that makes it hard to decompile.

s is the first, and only, local (so on the stack) variable in main(). main() is troublesome, as it's more or less a vararg-function if you read the C++ standard, and as you can see atleast IDA guesses that you have three arguments on the stack.

You use both int and long in your struct definition, which may or may not create either padding of the stack or masking in the generated code. It can also be one way where you declare it (main) and another way when passing it by value to a (leaf-)function.

And, foo() is a leaf-function, meaning it will have a red-zone on the stack that could possibly be used.

Try putting s on the heap instead, and you will probably see a very different result :)

What does the disassembly look like?