Questions tagged [calling-conventions]

As assembler languages have no built-in primitives to handle arguments passing while starting a procedure, most of assembler programs and compiler do agree on calling conventions to describe how to emulate arguments passing at the assembler level.

Filter by
Sorted by
Tagged with
0
votes
1answer
28 views

indirect return value address

I have usercall with calling convention I do not fully understand, it returns std::string but IDA recognize it as void. I've noticed that every calle reads from x8/w8 afterwards, from wikipedia; ...
0
votes
1answer
39 views

Why doesn't alloca_probe follow x64 calling convention?

I was practising reverse engineering on some Windows x64 applications when I came across this function: call alloca_probe This alloca_probe function has some strange implementation: EAX is used as ...
1
vote
1answer
95 views

Lack of arguments before a CALL instruction

I'm reversing an open-source windows application written in C++. I found in the disassembler the desired function that I'm trying to understand its behavior. In the code this function is defined as ...
7
votes
2answers
872 views

Register Calling Convention: written in stone, or in mud?

When disassembling an old Delphi 3 executable, I find some routines that pass arguments in registers EAX, EDX, and on the stack – but not in ECX! For those routines, ECX never gets set to a '...
1
vote
2answers
336 views

IDA Hex-Rays: How to SAFELY fix incorrect function declarations?

Update: The problem turned out to be more complex and complex because Hex-Rays incorrectly restores the stack after calls of stdcall functions from cdecl functions: .text:00403F2F 074 ...
2
votes
0answers
144 views

Show parameters stored in stack with __fastcall in IDA

This is an extension of my previous question. Here is a sample of assembly in x64 Windows executable using COM virtual function with __fastcall convention. mov [rsp+32], rdx ...
2
votes
2answers
1k views

Confused about the value of edx and eax registers used at function entry

I am currently using IDA to disassemble a keygen. In the first few lines of a specific function before the eax and edx registers are given a value, their contents are moved to stack variables var_40 ...
2
votes
1answer
107 views

What does __return_ptr do in IDA?

Let's say I have this program: struct A { int x; }; struct A func1(int z) { struct A result; result.x = z+1; return result; } int main() { return func1(1).x; } When compiling ...
4
votes
1answer
418 views

How to specify stack-based return values in IDA Pro (GoLang)

tl;dr: I am reversing a GoLang 1.10 executable, compiled for Windows. I am trying to make IDA correctly recognize the calling convention. Details: I am looking at an x86 executable where the ...
2
votes
2answers
274 views

What compiler uses a calling convention that uses EAX as the first argument, then pushes onto the stack?

I recently asked to question: How can an IA-32 program seemingly compiled with MSVC send its sole argument via EAX? After posting the question, I found that another function passed the first argument ...
1
vote
1answer
103 views

How can an IA-32 program seemingly compiled with MSVC send its sole argument via EAX? [duplicate]

A C++ program I'm looking at appears to have been compiled with MSVC. It links to MSVCR100.dll, contains MSVC's security cookie code and uses the Microsoft mangling scheme. However, one of the ...
6
votes
1answer
220 views

Returning value from faux DLL

A bit of background; I'm trying to inject my own code into the old game SimTower with the ultimate goal of reverse engineering it in the same way that OpenRCT2 was created. Since SimTower is ...
2
votes
1answer
377 views

How can I call a function that acts like __thiscall, except the caller cleans the stack?

I'm trying to call a function with a function pointer in C++, but I can't find any convention that fits it. Its stack arguments are (right to left): func(float x, float y, int unk); ...but ecx needs ...
6
votes
3answers
6k views

IDA Pro function stack frame view

IDA Pro displays certain buffer or padding above (at lower addresses) local variables in stack frame view. For instance: Example 1. The following screen shot of stack frame view shows 12 bytes (...
12
votes
3answers
7k views

Which calling convention to use for EAX/EDX in IDA

I have some code (I assume Delphi) which uses only the EAX and EDX register for passing the arguments (and of course the stack if more are required). I looked which calling conventions would match, ...
4
votes
1answer
857 views

How to find the stack and other info in a memory dump? ARM

I'm reversing a closed platform to try gain execution using an exploit, a stack overflow. I've been told that (since there are no debuggers) the best way is to use RAM dumps (that I have) to try ...
1
vote
0answers
231 views

IDA Pro .til files backward compatibility

I read in the IDA Pro Book that you may use till.exe to parse header files and create standalone .til files which can be made globally available by copying them into <IDADIR>/til. This ...
2
votes
1answer
106 views

Calling a function outside the module

Is it possible that a module will have a call to a function in a different module? I know about import tables and delayed import tables. But are there any more options to do so? How will it look in ...
1
vote
2answers
690 views

Confused about calling convention in a Windows kernel mode driver

Reverse engineering a kernel mode driver (in its 32-bit x86 incarnation) I stumbled over what seems to be an odd calling convention. For a driver I'd expect to see __cdecl, __fastcall and __stdcall in ...
1
vote
1answer
491 views

Call a thiscall method from an injected DLL [closed]

I am trying to create a (32-bit) DLL to inject into a target process. My DLL should call a function inside the original process. So far nothing special, however the target function is using the ...
3
votes
1answer
929 views

Why is this _DllMainCRTStartup calling convention not as expected?

I've been looking around at various DLLs on my computer lately and I thought a decent place to start was the image entry point (typically _DllMainCRTStartup). From what I can see of the CRT source ...
3
votes
1answer
2k views

How to create function prototype that recognizes arguments passed by reference in IDA Pro

I have a function prototype generated by IDA that looks like this: However, it looks like there is a struct being referenced at the instruction lea edi, [esp+290h+var_240]. I would like IDA to ...
3
votes
2answers
731 views

x86_64 Calling convention

I am slightly confused by some x86_64 linux code here (generated by llvm): # BB#0: # %entry push RBP .Ltmp21: .cfi_def_cfa_offset 16 .Ltmp22: ....
2
votes
2answers
616 views

How to programmatically identify binary calling convention?

I have a problem where I'm instrumenting stripped binaries; I don't know the start of main(). But there's always an init(), and init() calls libc_start_main(), which receives a pointer to main. If I ...
2
votes
2answers
817 views

Change calling convention from IDAPython?

I have a 64-bit Mach-O sample that uses the System V AMD64 ABI calling convention (rdi, rsi, rdx, rcx,...); but IDA thinks all the functions are using cdecl. I'm new to IDA scripting but didn't find ...
8
votes
1answer
2k views

What x86 calling convention passes first parameter via ESI?

I am looking at some x86 code, which I believe was built using a Microsoft tool chain, and am trying to figure out the calling convention used during this call: push esi ; save ESI (it gets ...
8
votes
2answers
1k views

Identifying variable args function

How would a C variable argument function such as printf(char* format, ...) look like when disassembled? Is it always identified by calling convention, or are there more ways to identify it?