We changed our privacy policy. Read more.
24

you can add a function type by editing it (Key Y) and adding the name. I wrote a page to remind me about calling conventions at ASM level. Introduction the original call is myfunc(0,1,2,3,4). standard order is first argument is pushed last. standard stack adjusting is 'callee cleanup' - after returning, the stack should be without its calling arguments. ...


12

There is no "official" calling convention that works like that. What you're seeing is most likely the result of Link-time Code Generation, also known as LTO (Link-time optimization) or WPO (Whole program optimization). When it is enabled, the optimization and code generation is done at link time, when the compiler has access to the code of whole program and ...


10

If you run into a calling convention which is not covered by any of the standard calling conventions you can use the __usercall or __userpurge calling convention which allows you to specify which arguments are passed where. The syntax is return_type __usercall function_name<registers>(arg0_type arg0<registers>, arg1_type arg1<registers>, ....


7

My guess is that the function signature should be something like: int __usercall Call_HTTP@<eax>(int x, int y, void* http_object@<edi>); __usercall means the calling convention for the function is not a standard one (like stdcall, cdecl, etc.) as the function passes two arguments on stack and one in edi. @<eax> : function returns a value ...


7

This is probably a program compiled with "Whole Program Optimization" or "Link-time code generation". From MSDN: When /LTCG is used to link modules compiled with /Og, /O1, /O2, or /Ox, the following optimizations are performed: Cross-module inlining Interprocedural register allocation (64-bit operating systems only) Custom calling ...


7

If the compiler can prove that it has all call sites for a given function under its control then it can discard conventions and arrange things around to its liking. Microsoft's C/C++ compiler has been doing this for decades in connection with link-time code generation and profile-guided optimisation, especially internal copies of the compiler like the one ...


6

Your faux function is not fixing up the stack on the epilog of the function. It should have pop ds pop bp dec bp at the end by the calling convention. If you look at the original function it fixes up the stack correctly. Let's say the stack pointer had the value of 1000h coming into the function (after CS:IP is already pushed on the stack for the far ...


5

The only idea I have is to compare the dumps. The places that are same in all dumps are code or read only data. The places that are changing from dump to dump are either stack or section like .bss. After finding places that are not changing I'd try to disassemble these places in order to divide between code and data. I think that the places with the code ...


5

With global optimisation (link-time code generation) the compiler is free to ignore the declared calling convention for all functions that aren't externally visible - that is, neither exported nor otherwise pinned by having their addresses taken and passed to external code. This can make the resulting binaries difficult to analyse since even the well-known ...


5

Delphi and Borland C++ Builder use EAX, EDX and ECX for the first three arguments in their variant of the __fastcall calling convention. So if you choose "Delphi" or "C++ Builder" in Options-Compiler, you can just use __fastcall in the function prototype - no need to resort to __usercall.


3

The "problem" with your example is that the structure is too small (four bytes), so it fits in a register and is not actually passed on the stack. From the Itanium C++ ABI (used by most GCC implementations): A type is considered non-trivial for the purposes of calls if: it has a non-trivial copy constructor, move constructor, or destructor, or ...


3

So, your question is mainly about this part of the code (if I do not care about the second call which is similar anyway): mov EDI, .L.urandom mov ESI, 8 call open In fact, this is perfectly okay to go through ESI and EDI as it is done here, because the value that are transfered are 32 bits values... and, anyway, in amd64, when manipulating 32 bits ...


3

Since you mention __libc_start_main it seems you need to worry only about Linux. In such case, the calling convention is known and fixed. In most cases it will be SystemV ABI or similar.


3

Use idc.SetType (see documentation here ) It works exactly the same as pressing y on function and defining the type, but you have to add semicolon to the end of the type definition.


2

We can see from this list that IA-32's Delphi/Free Pascal calling convention is the register calling convention. My guess is you're dealing with a binary that's the result of something like this: How to call a function using Delphi's register calling conventions from Visual C++? To partially quote the top-voted answer: Delphi's register calling ...


2

Sounds like you have a variadic __thiscall function (printf-like). Those use ecx for this but the remaining arguments are pushed onto the stack and the caller cleans it up.


2

The decompiler decided that ecx is used by sub_431C00 because of push ecx at the beginning of the function which fills the stack slot later used by the variable phkResult, so it may look as if the initial value of phkResult is taken from ecx: .text:00431C00 push ecx .text:00431C01 lea eax, [esp+4+phkResult] .text:...


2

Welcome to RE.SE! How to understand what the problem is? When the function is decompiled, IDA checks for its dependencies via dataflow analysis. Basically, it asks which values have to be defined in order to make this function work. When it encounters the first line of the function (push ecx) IDA noticed that the value of ecx is used before it was ...


2

Yep, I think I've seen a similar case before. It happens very rarely, but because you can do some stunts at the C preprocessor level as well as at the linker level, this happens to work. In this case I looked it up in the 3790.1830 DDK (Windows 2003 Server) and several newer WDKs and SDKs. Windows 2000/XP target (3790.1830/inc/w2k/winbase.h and 3790.1830/...


2

From https://github.com/Siguza/ios-resources/blob/master/bits/arm64.md x8 pointer to where to write the return value if >128 bits, otherwise scratch register So it seems x8 is used to pass an address at which the return value will land if it's too big, rather than the return value directly, hence indirect I suppose. I couldn't find official ...


2

Calling conventions are compiler specific, meaning they are specified by the compiler manufacturer like Microsoft, and not by the 8086 manufacturer like Intel. In case you program in Assembler or build your own compiler, you can apply your own method to pass function parameters. Nevertheless, in the C/C++ world, there are mainly the __cdecl and the __stdcall ...


1

First make sure you are using correct calling convention, in 64-bit Windows most APIs expect parameters via registers not push. An example of calling convention for 64-bit Windows: lea r9d, uType ; uType lea r8, Caption ; Caption lea rdx, Text ; Text mov rcx, hWnd ; hWnd Call cs:MessageBoxW This is covered in more ...


1

I have found the solution to this problem, which is quite simple. I got confused with the hooked function signature and the __fastcall trick to get a non member function to work with the __thiscall convention. In my initial post, the two first arguments in the function signatures are the one passed using ecx and edx, but I forgot about one of the pointer ...


1

This is not a standard function but a compiler helper, used by the compiler to perform some necessary housekeeping (allocate extra stack space) so it doesn’t have to follow the ABI for “normal” functions. Because it is called in the prolog of the parent function while the user code has not started executing yet, special care needs to be taken. EAX use ...


1

Seems like it has 4 arguments, and they are passed through stack, yet the more lines of disassembly above E42D6A needed to make sure. Are those mov and lea instruction confusing you? If you've expected to have 4 push instruction in a row, then let me disappoint you: compiler can insert as many instructions as it likes between the push instructions. And let ...


1

I have been in touch with IDA support and unfortunately, right now, there does not seem to be a good way to do this. IDA assumes in many cases that the return value of a function must be stored in a register. This is also the reason for the crash when trying to overrule this inherent assumption with scattered arguments.


1

Most likely it's just LTCG/LTO, especially if the function in question is only never called externally. Using it can result in the following: Cross-module inlining Interprocedural register allocation (64-bit operating systems only) Custom calling convention (x86 only) Small TLS displacement (x86 only) Stack double alignment (x86 only) Improved memory ...


1

Many non-Microsoft compilers pass the first argument in EAX. From https://en.wikipedia.org/wiki/X86_calling_conventions: optlink The three lexically first (leftmost) arguments are passed in EAX, EDX, and ECX... ... Borland register Evaluating arguments from left to right, it passes three arguments via EAX, EDX, ECX. ... Watcom register ...


1

What error you get? Try something like this: class HashClass { public: byte data[1024]; LPVOID HashFunc(float, char, char); }; typedef LPVOID (HashClass::*HASHFUNC)(float, char, char); HASHFUNC *HashFunc = (HASHFUNC *) 0xD34DB33F; void CallHashFunction() { HashClass *hashClass = new HashClass(); (*hashClass.*HashFunc)(1.0f, 'a', 'b'); ...


1

There is no need to do anything. For x64 Mach-O and ELF files, cdecl is the same as fastcall and means SysV AMD64 ABI.


Only top voted, non community-wiki answers of a minimum length are eligible