3

Introduction

I compiled a simple executable with Visual Studio in x64 Windows. Source code:

long test(int a, int b, int c, int d, int e, int f, int g, int h, int i, int j) {
  printf("%d %d %d", a, b, c);
  return 0x0123456789acdef;
}

int main() {
  test(1,2,3,4,5,6,7,8,9,10);
}

If you compile the executable with GCC, then you get what you'd expect: one function call in main (it calls test) and one function call in test (it calls printf).

However, if you compile the executable with Visual Studio on an x64 Windows machine, then in both main and test, an additional call is made to some function. Let's call that function mystery. I'd like to know what the mystery function is for.

Code

Here is the disassembly of the main function. I have added a comment to show where the mystery function is called.

int main (int argc, char **argv, char **envp);
; var int64_t var_c8h @ rbp+0xc8
; var int64_t var_20h @ rsp+0x20
; var int64_t var_28h @ rsp+0x28
; var int64_t var_30h @ rsp+0x30
; var int64_t var_38h @ rsp+0x38
; var int64_t var_40h @ rsp+0x40
; var int64_t var_48h @ rsp+0x48
; var int64_t var_50h @ rsp+0x50

push rbp
push rdi
sub rsp, 0x118
lea rbp, [var_50h]
lea rcx, [0x1400c1003]
call fcn.140034cf1         ; Mystery function here!
mov dword [var_48h], 0xa
mov dword [var_40h], 9
mov dword [var_38h], 8
mov dword [var_30h], 7
mov dword [var_28h], 6
mov dword [var_20h], 5
mov r9d, 4
mov r8d, 3
mov edx, 2
mov ecx, 1
call fcn.140033e73
xor eax, eax
lea rsp, [var_c8h]
pop rdi
pop rbp
ret

The same mystery function is also called in the test function:

test (int64_t arg1, int64_t arg2, int64_t arg3, int64_t arg4);
; var int64_t var_c8h @ rbp+0xc8
; var int64_t var_20h_2 @ rsp+0x20
; var int64_t var_8h @ rsp+0x100
; var int64_t var_10h @ rsp+0x108
; var int64_t var_18h @ rsp+0x110
; var int64_t var_20h @ rsp+0x118
; arg int64_t arg1 @ rcx
; arg int64_t arg2 @ rdx
; arg int64_t arg3 @ r8
; arg int64_t arg4 @ r9

mov dword [var_20h], r9d   ; arg4
mov dword [var_18h], r8d   ; arg3
mov dword [var_10h], edx   ; arg2
mov dword [var_8h], ecx    ; arg1
push rbp
push rdi
sub rsp, 0xe8
lea rbp, [var_20h_2]
lea rcx, [0x1400c1003]
call fcn.140034cf1         ; Mystery function here!!!
mov r9d, dword [var_18h]
mov r8d, dword [var_10h]
mov edx, dword [var_8h]
lea rcx, str._d__d__d      ; 0x14009ff98 ; "%d %d %d"      
call fcn.1400335b3
mov eax, 0x789acdef
lea rsp, [var_c8h]
pop rdi
pop rbp
ret

Here are the contents of the mystery function:

├ 60: mystery (int64_t arg1);
│           ; var int64_t var_20h @ rsp+0x20
│           ; var int64_t var_8h @ rsp+0x40
│           ; arg int64_t arg1 @ rcx
│           0x1400387d8      48894c2408     mov qword [var_8h], rcx    ; arg1
│           0x1400387dd      4883ec38       sub rsp, 0x38
│           0x1400387e1      488b442440     mov rax, qword [var_8h]
│           0x1400387e6      4889442420     mov qword [var_20h], rax
│           0x1400387eb      488b442440     mov rax, qword [var_8h]
│           0x1400387f0      0fb600         movzx eax, byte [rax]
│           0x1400387f3      85c0           test eax, eax
│       ┌─< 0x1400387f5      7418           je 0x14003880f
│       │   0x1400387f7      833d06fa0700.  cmp dword [0x1400b8204], 0 ; [0x1400b8204:4]=0
│      ┌──< 0x1400387fe      740f           je 0x14003880f
│      ││   0x140038800      ff15fa770800   call qword [sym.imp.KERNEL32.dll_GetCurrentThreadId] ; [0x1400c0000:8]=0xc0738 reloc.KERNEL32.dll_GetCurrentThreadId ; "8\a\f" ; DWORD GetCurrentThreadId(void)
│      ││   0x140038806      3905f8f90700   cmp dword [0x1400b8204], eax ; [0x1400b8204:4]=0
│     ┌───< 0x14003880c      7501           jne 0x14003880f
│     │││   0x14003880e      90             nop
│     │││   ; CODE XREFS from mystery @ 0x1400387f5, 0x1400387fe, 0x14003880c
│     └└└─> 0x14003880f      4883c438       add rsp, 0x38
└           0x140038813      c3             ret

Notes

The call to KERNEL32.dll_GetCurrentThreadId in the mystery function seemed interesting, but I couldn't find any information about functions, which get added by the compiler and call GetCurrentThreadId.

The question arose because I was studying the Microsoft x64 calling convention and I noticed that the mystery function didn't seem to use "home space" (also known as "shadow space", 32 bytes of space on the stack that's assigned to each function). I wanted to know what that function is and why home space wasn't assigned to it.

So the question is, what is the mystery function?

9

The "mystery" function you refer to is called __CheckForDebuggerJustMyCode and is inserted by MSVC when /JMC (Just My Code debugging) option is enabled. From docs:

/JMC (Just My Code debugging)

Specifies compiler support for native Just My Code debugging in the Visual Studio debugger. This option supports the user settings that allow Visual Studio to step over system, framework, library, and other non-user calls, and to collapse those calls in the call stack window.

[...]

The Visual Studio Just My Code settings specify whether the Visual Studio debugger steps over system, framework, library, and other non-user calls. The /JMC compiler option enables support for Just My Code debugging in your native C++ code. When /JMC is enabled, the compiler inserts calls to a helper function, __CheckForDebuggerJustMyCode, in the function prolog. The helper function provides hooks that support Visual Studio debugger Just My Code step operations.

To verify that the "mystery" function indeed corresponds to __CheckForDebuggerJustMyCode, you can put the breakpoint in Visual Studio in the main's first line, press F5 to start debugging, and then ctrl + alt + d to show window with the assembly generated by the compiler. You will get something like this:

main's disassembly

2

provide full code compiler version and compiler / linker arguments

here i checked it with msvc 2017 x64 and i don't find any mystery function

also i ran it on Compiler Explorer with several versions of msvc x64 and they dont spit it either

:\>cl 2>&1 | head -n 1
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27045 for x64

:\>cat myst.cpp
#include <stdio.h>
long test(int a, int b, int c, int d, int e, int f, int g, int h, int i, int j) {
  printf("%d %d %d", a, b, c);
  return 0x0123456789acdef;
}

int main() {
  test(1,2,3,4,5,6,7,8,9,10);
}

:\>cl /Zi /W4 /analyze /Od /EHsc /nologo myst.cpp /link /release
myst.cpp
myst.cpp(4): warning C4305: 'return': truncation from '__int64' to 'long'
myst.cpp(4): warning C4309: 'return': truncation of constant value
myst.cpp(2): warning C4100: 'j': unreferenced formal parameter
myst.cpp(2): warning C4100: 'i': unreferenced formal parameter
myst.cpp(2): warning C4100: 'h': unreferenced formal parameter
myst.cpp(2): warning C4100: 'g': unreferenced formal parameter
myst.cpp(2): warning C4100: 'f': unreferenced formal parameter
myst.cpp(2): warning C4100: 'e': unreferenced formal parameter
myst.cpp(2): warning C4100: 'd': unreferenced formal parameter

:\>cdb -c "uf myst!main;q" myst.exe | awk /Reading/,/quit/
0:000> cdb: Reading initial command 'uf myst!main;q'
myst!main:
00007ff6`538a1040 4883ec58        sub     rsp,58h
00007ff6`538a1044 c74424480a000000 mov     dword ptr [rsp+48h],0Ah
00007ff6`538a104c c744244009000000 mov     dword ptr [rsp+40h],9
00007ff6`538a1054 c744243808000000 mov     dword ptr [rsp+38h],8
00007ff6`538a105c c744243007000000 mov     dword ptr [rsp+30h],7
00007ff6`538a1064 c744242806000000 mov     dword ptr [rsp+28h],6
00007ff6`538a106c c744242005000000 mov     dword ptr [rsp+20h],5
00007ff6`538a1074 41b904000000    mov     r9d,4
00007ff6`538a107a 41b803000000    mov     r8d,3
00007ff6`538a1080 ba02000000      mov     edx,2
00007ff6`538a1085 b901000000      mov     ecx,1
00007ff6`538a108a e871ffffff      call    myst!test (00007ff6`538a1000)
00007ff6`538a108f 33c0            xor     eax,eax
00007ff6`538a1091 4883c458        add     rsp,58h
00007ff6`538a1095 c3              ret
quit:

:\>cdb -c "uf myst!test;q" myst.exe | awk /Reading/,/quit/
0:000> cdb: Reading initial command 'uf myst!test;q'
myst!test:
00007ff6`538a1000 44894c2420      mov     dword ptr [rsp+20h],r9d
00007ff6`538a1005 4489442418      mov     dword ptr [rsp+18h],r8d
00007ff6`538a100a 89542410        mov     dword ptr [rsp+10h],edx
00007ff6`538a100e 894c2408        mov     dword ptr [rsp+8],ecx
00007ff6`538a1012 4883ec28        sub     rsp,28h
00007ff6`538a1016 448b4c2440      mov     r9d,dword ptr [rsp+40h]
00007ff6`538a101b 448b442438      mov     r8d,dword ptr [rsp+38h]
00007ff6`538a1020 8b542430        mov     edx,dword ptr [rsp+30h]
00007ff6`538a1024 488d0d15c30400  lea     rcx,[myst!__xt_z+0x8 (00007ff6`538ed340)]
00007ff6`538a102b e8d0000000      call    myst!printf (00007ff6`538a1100)
00007ff6`538a1030 b8efcd9a78      mov     eax,789ACDEFh
00007ff6`538a1035 4883c428        add     rsp,28h
00007ff6`538a1039 c3              ret
quit:

edit as bart1e pointed out in his answer it was debug_just_my_code the argument /jmc wasn't documented in cl /? in vs 2017 but has been implemented

:\>cl /? >args.txt
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27045 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.


:\>grep -ir jmc args.txt

:\>grep -ir jm args.txt

:\>grep -ir j args.txt
/Fm[file] name map file                 /Fo<file> name object file
/Fm: <file> name map file               /Fo: <file> name object file
/P preprocess to file                   /Fx merge injected code to file
    c++latest - latest draft standard (feature set subject to change)
/permissive[-] enable some nonconforming code to compile (feature set subject to change) (on by default)
  alignedNew[-]         enable C++17 alignment of dynamically allocated objects (on by default)
/Zp[n] pack structs on n-byte boundary  /Zl omit default library name in .OBJ
/bigobj generate extended object format /c compile only, no link
/J default char type is unsigned
/Yd put debug info in every .OBJ        /Yl[sym] inject .PCH ref for debug lib

:\> 

passing jmc and resulting just my code function call addition

:\>cdb -c "uf myst!main;uf myst!__CheckForDebuggerJustMyCode;q" myst.exe | awk /Reading/,/quit/
0:000> cdb: Reading initial command 'uf myst!main;uf myst!__CheckForDebuggerJustMyCode;q'

myst!main:
00007ff7`3c031050 4883ec58        sub     rsp,58h
00007ff7`3c031054 488d0da84f0600  lea     rcx,[myst!_DebuggerCurrentSteppingThreadId <PERF> (myst+0x66003) (00007ff7`3c096003)]
00007ff7`3c03105b e844010000      call    myst!__CheckForDebuggerJustMyCode (00007ff7`3c0311a4)
00007ff7`3c031060 c74424480a000000 mov     dword ptr [rsp+48h],0Ah
00007ff7`3c031068 c744244009000000 mov     dword ptr [rsp+40h],9
00007ff7`3c031070 c744243808000000 mov     dword ptr [rsp+38h],8
00007ff7`3c031078 c744243007000000 mov     dword ptr [rsp+30h],7
00007ff7`3c031080 c744242806000000 mov     dword ptr [rsp+28h],6
00007ff7`3c031088 c744242005000000 mov     dword ptr [rsp+20h],5
00007ff7`3c031090 41b904000000    mov     r9d,4
00007ff7`3c031096 41b803000000    mov     r8d,3
00007ff7`3c03109c ba02000000      mov     edx,2
00007ff7`3c0310a1 b901000000      mov     ecx,1
00007ff7`3c0310a6 e855ffffff      call    myst!test (00007ff7`3c031000)
00007ff7`3c0310ab 33c0            xor     eax,eax
00007ff7`3c0310ad 4883c458        add     rsp,58h
00007ff7`3c0310b1 c3              ret


myst!__CheckForDebuggerJustMyCode:
00007ff7`3c0311a4 48894c2408      mov     qword ptr [rsp+8],rcx
00007ff7`3c0311a9 4883ec38        sub     rsp,38h
00007ff7`3c0311ad 488b442440      mov     rax,qword ptr [rsp+40h]
00007ff7`3c0311b2 4889442420      mov     qword ptr [rsp+20h],rax
00007ff7`3c0311b7 488b442440      mov     rax,qword ptr [rsp+40h]
00007ff7`3c0311bc 0fb600          movzx   eax,byte ptr [rax]
00007ff7`3c0311bf 85c0            test    eax,eax
00007ff7`3c0311c1 7418            je      myst!__CheckForDebuggerJustMyCode+0x37 (00007ff7`3c0311db)

myst!__CheckForDebuggerJustMyCode+0x1f:
00007ff7`3c0311c3 833d7ef4050000  cmp     dword ptr [myst!__DebuggerCurrentSteppingThreadId (00007ff7`3c090648)],0
00007ff7`3c0311ca 740f            je      myst!__CheckForDebuggerJustMyCode+0x37 (00007ff7`3c0311db)

myst!__CheckForDebuggerJustMyCode+0x28:
00007ff7`3c0311cc ff152ebe0400    call    qword ptr [myst!_imp_GetCurrentThreadId (00007ff7`3c07d000)]
00007ff7`3c0311d2 390570f40500    cmp     dword ptr [myst!__DebuggerCurrentSteppingThreadId (00007ff7`3c090648)],eax
00007ff7`3c0311d8 7501            jne     myst!__CheckForDebuggerJustMyCode+0x37 (00007ff7`3c0311db)

myst!__CheckForDebuggerJustMyCode+0x36:
00007ff7`3c0311da 90              nop

myst!__CheckForDebuggerJustMyCode+0x37:
00007ff7`3c0311db 4883c438        add     rsp,38h
00007ff7`3c0311df c3              ret
quit:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.