# Calling Conventions

## What is Calling Convention? <a href="#what-is-calling-convention" id="what-is-calling-convention"></a>

The term “calling convention” describes:

* How arguments passed to the function and how values are returned from functions.
* Whether the caller or the callee will cleans the stack.

There are different conventions based on processor, OS and language.&#x20;

Also, not every compiler implements the “calling conventions” the same way. Therefore, only the standard 4 calling conventions will be discussed in this article.

* \_\_cdecl
* \_\_stdcall
* \_\_fastcall
* \_\_thiscall

Also, developers can sometimes use custom calling conventions. For detailed information on this subject, you can subscribe to [OALabs from Patreon](https://www.patreon.com/oalabs) and view the \_\_usercall tutorial. They have very good videos on this subject.

## \_\_cdecl <a href="#cdecl" id="cdecl"></a>

\_\_cdecl (which stands for C declaration) is the default calling convention in C and C++. Therefore, you will come across this definition many times while reading assembly in tools such as IDA and Ghidra.

Characteristics of `__cdecl:`

1. Arguments are passed `on the stack in reverse order` (pushed right-to-left),
2. `Caller` cleans the stack.
3. The return value is stored in `EAX`.

Let’s say we have a function called “sumNumbers” and it takes 5 integer arguments, then returns the sum of these numbers.

```cpp
#include <iostream>

int sumNumbers(int number1, int number2, int number3, int number4, int number5)
{
    int numSum = number1 + number2 + number3 + number4 + number5;
    return numSum;

}
int main()
{
    int numberSum = 0;
    numberSum = sumNumbers(1, 2, 3, 4, 5);
    printf("Result: %d",numberSum);
}
```

After compiling this code, when we open it in a tool like Ghidra or IDA, it will look like below:

<figure><img src="https://4247064012-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FFLaJdzZGSq1DpczuSySw%2Fuploads%2FvoJe5MbkrgGFDgJuW7Zy%2Fimage.png?alt=media&#x26;token=77811b09-b62f-414e-873e-5bc20b610ffa" alt=""><figcaption></figcaption></figure>

* As seen in the first box, the arguments were passed to the stack in reverse order, then the function called. Which is the first key point to recognize \_\_cdecl. `(1)`
* And in the second box, `0x14 (20) is added to the ESP`. As 5 arguments are sent to the stack and each argument takes up 4 bytes, `caller cleans the stack` 20 byte. `(2)`
* In \_\_cdecl, you will usually see the line `add esp,<area to be cleaned>,` 1 step after the function call.
* And as seen in the last box, the `return value in EAX` is passed to the printf function to be printed. `(3)`
* Also you can see that, the reverse engineering tools usually recognize calling functions. (Blue Underline in screenshot.) But we still won’t trust the information there too much. We will examine the reason specifically under the “Why are calling conventions important for reverse engineering ? “ part.

## \_\_stdcall <a href="#stdcall" id="stdcall"></a>

\_\_stdcall is the standard calling convention for Win32 API calls. \_\_stdcall characteristics almost identical with \_\_cdecl calling convention.

Characteristics of `__stdcall:`

1. Arguments are passed `on the stack in reverse order` (pushed right-to-left), → Same as \_\_cdecl,
2. Callee cleans the stack.
3. The return value is stored in `EAX`. → Same as \_\_cdecl.

As you can see, the only difference between \_\_cdecl and \_\_stdcall is that, in \_\_cdecl calling convention caller cleans the stack, while in \_\_stdcall calling convention callee cleans the stack.

In this example we will look at the InternetOpenA API, which takes 5 arguments:

```c
HINTERNET InternetOpenA(
  [in] LPCSTR lpszAgent,
  [in] DWORD  dwAccessType,
  [in] LPCSTR lpszProxy,
  [in] LPCSTR lpszProxyBypass,
  [in] DWORD  dwFlags
);
```

Let’s compile the code below for analyzing InternetOpenA WIN32 API.

```cpp
#include <iostream>

#pragma comment(lib, "wininet.lib")
#include <WinSock2.h>
#include <wininet.h>

int main()
{
    std::string RESULT{};
    const int size = 4096;
    char buf[size];
    DWORD length;

    HINTERNET internet = InternetOpenA("Mozilla/5.0", INTERNET_OPEN_TYPE_DIRECT, NULL, NULL, NULL);
    if (!internet)
        ExitProcess(EXIT_FAILURE);

    HINTERNET response = InternetOpenUrlA(internet, "http://www.google.com", NULL, NULL, NULL, NULL);
    if (!response) {
        InternetCloseHandle(internet);
        ExitProcess(EXIT_FAILURE);
    }
    InternetReadFile(response, buf, size, &length);
    InternetCloseHandle(response);
    InternetCloseHandle(internet);

    std::cout << buf << std::endl;
    return 0;
}
```

After compiling this code, when we open it in a tool like Ghidra or IDA, it will look like below.

<figure><img src="https://tufan-gungor.github.io/assets/img/calling_conventions/Untitled%201.png" alt=""><figcaption></figcaption></figure>

* As seen in the first box, the 5 parameters requested by the InternetOpenA API were `passed to the stack in reverse order`. `(1)`
* In the second box, after the InternetOpenA `result is returned in EAX,` it is moved to the \[ebp+hInternet] address and checked if it is equal to 0 or not. `(3)`
* As a result, the caller did not take any action to clear the stack. Because the `callee (the InternetOpenA function itself) is responsible for stack cleaning`. `(2)`

As we showed on Ghidra in the previous example, we can see the calling convention of this function on IDA as well. (Hover mouse over the function name.)

<figure><img src="https://4247064012-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FFLaJdzZGSq1DpczuSySw%2Fuploads%2FKIHRbs8OehE5Jth87DRK%2Fimage.png?alt=media&#x26;token=3f148573-ffef-4192-8aaf-3c1aba6434fa" alt=""><figcaption></figcaption></figure>

## \_\_fastcall <a href="#fastcall" id="fastcall"></a>

The main difference of \_\_fastcall is that the initial arguments are passed to the registers instead of pushing to the stack. It’s faster to keep data in registers than in memory, so it’s called `FAST` call.

Characteristics of `__fastcall`:

* First two or three parameters will be passed in the registers EDX, ECX or EAX, and additional parameters are passed on to the stack.
* The return value is stored in `EAX`.
* `Callee` cleans the stack.

_The first two DWORD or smaller arguments that are found in the argument list from left to right are passed in ECX and EDX registers; all other arguments are passed on the stack from right to left._ `(Microsoft)`

Let’s compile the code below and analyze it in IDA.

```cpp
#include <iostream>

int __fastcall TestFunction(int num1, int num2, int num3, int num4, int num5)
{
    int nSum = 0;
    nSum = num1 + num2 + num3 + num4 + num5;
    return nSum;
}

int main()
{
    int nSum = 0;
    nSum = TestFunction(10, 20, 30, 40, 50);
    printf("Result: %d", nSum);
    return 0;
}
```

<figure><img src="https://4247064012-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FFLaJdzZGSq1DpczuSySw%2Fuploads%2FopOFPlebY6jfLqyOQT3K%2Fimage.png?alt=media&#x26;token=c1a195bb-9b3b-4212-a3bd-48bb6a0904f5" alt=""><figcaption></figcaption></figure>

* By looking at the first box, you can see that the `first 2 parameters are passed in the ECX and EDX` and the `other parameters are passed on to the stack`. `(1)`
* In the second box, `return value stored in EAX` and passed to the printf function. `(2)`
* We told that the `callee is responsible for stack cleaning (not registers)`, so in the last box you can see `caller` cleaning 8 bytes. 8/4=2 parameters. Which they are `ECX` and `EDX`.

To see how callee handles stack cleaning, let’s jump into that function:

<figure><img src="https://4247064012-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FFLaJdzZGSq1DpczuSySw%2Fuploads%2Fo8fbnl5lHNmqdyp0qZfv%2Fimage.png?alt=media&#x26;token=d7137b94-d737-4366-a830-ca31436f5e18" alt=""><figcaption></figcaption></figure>

* Since the first 2 arguments cleaned from ECX and EDX by caller, there is only 3 arguments left to be cleaned from stack. You can see in the box, `callee cleans up the stack` by using an operand to the `retn` instruction of `0Ch` (12). So the extra `12-bytes` (12/4 = 3 arguments) to be cleaned up `from stack` during the return operation.
* You can also see that IDA recognizes that this function uses the \_\_fastcall calling convention.

> \_\_fastcall is the default calling convention for Windows x64 Application Binary Interface (ABI). Unlike x32, it sends the first four parameters to the registers `rcx`, `rdx`, `r8` and `r9`, and the remaining parameters to the `stack`.

_Integer arguments are passed in registers RCX, RDX, R8, and R9. Floating point arguments are passed in XMM0L, XMM1L, XMM2L, and XMM3L. 16-byte arguments are passed by reference. Parameter passing is described in detail in Parameter passing. These registers, and RAX, R10, R11, XMM4, and XMM5, are considered volatile, or potentially changed by a callee on return._ `(Microsoft)`

## \_\_thiscall <a href="#thiscall" id="thiscall"></a>

The Microsoft-specific `__thiscall` calling convention is used on C++ class member functions on the x86 architecture. (passing the `“this”` object)

Characteristics of `__thiscall`:

* Arguments are passed `on the stack in reverse order` (pushed right-to-left).
* The `“this”` object pointer is passed via `ECX` register`.`
* `Callee` is responsible for stack cleanup.

Let’s compile a simple code to analyze \_\_thiscall.

```cpp
#include <iostream>
using namespace std;

class Numbers {
public:
    int sumNumbers(int num1, int num2);
};

int Numbers::sumNumbers(int num1,int num2) {
    int nSum = 0;
    nSum = num1 + num2;
    return nSum;
}

int main() {
    Numbers myObj; // Create an object of Numbers
    cout << myObj.sumNumbers(200,100); // Call the method with an argument
    return 0;
}
```

<figure><img src="https://4247064012-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FFLaJdzZGSq1DpczuSySw%2Fuploads%2F03Dm1m9JQzdK32Lsjurl%2Fimage.png?alt=media&#x26;token=1f8a6697-3d35-4648-8643-13cb8ab45187" alt=""><figcaption></figcaption></figure>

* Parameters (100,200) are `passed to the stack in reverse order.` Actually, we called as (200,100)
* The `“this”` object pointer stored in `ECX.`

To see where the stack cleaning job is done, let’s jump into the function.

<figure><img src="https://4247064012-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FFLaJdzZGSq1DpczuSySw%2Fuploads%2Fsylpk8HsqhRf2bbfeRD2%2Fimage.png?alt=media&#x26;token=ec0b4c6a-43a1-426d-99a8-0fca4312ff5c" alt=""><figcaption></figcaption></figure>

* You can see in the box, `callee cleans up the stack` by using an operand to the `retn` instruction of `8`. So the extra `8-bytes` (8/4 = 2 arguments) to be cleaned up `from stack` during the return operation.
* You can also see that IDA recognizes `__thiscall.`
