Injecting C++ code, while also maximizing compatibility with other injections

Question

Using the New Vegas Script Extender C++ API for Fallout New Vegas, I am able to write to the process' memory directly at runtime through a .dll plugin that's loaded by the NVSE injector. But it is not a debugger, like CheatEngine, so there's no automatic handling of inserting new code to an existing assembly function.

Beginning of the assembly function, TryOpenPipboy, I want to inject code into: https://i.stack.imgur.com/xIqmA.png

EquipPlayerPipboy() function I want to inject: https://i.stack.imgur.com/Gaujf.png

The game has a loop called MainUILoop which runs every frame, and when the player presses tab without holding it down, TryOpenPipboy is called.

My guess is that I need to somehow create a new empty block of memory, edit a copy of the original TryOpenPipboy function, then replace all TryOpenPipboy calls with my modified variant. But that would make a hard incompatibility with any other NVSE plugins that also modify the MainUILoop in any way, as well as any that edit TryOpenPipboy.

Edit: Another possible way would be to change the jmp assembly to my C++ function, then call TryOpenPipboy from my function. That would mean TryOpenPipboy is still called, but is unmodified, increasing potential compatibility.

So how should I go about doing this?

Answer 1

If you just want to run code before or after TryOpenPipboy is called, you can overwrite the prologue of the function with a jmp to your function. Before you overwrite it, copy the instructions in the prologue to a new memory buffer, and add a jump to the rest of the function on the end, this is called a trampoline. You can use the trampoline to call the original function.

Before:

:originalFunction
push ebp ;55
mov ebp, esp ;89 E5
sub esp, 0x40 ;83 EC 40
;...
ret

After patching:

:originalFunction
jmp yourFunction ;?? ?? ?? ?? ??
nop ;this used to be the 0x40 byte at the end
:trampolinetarget
;...
ret

yourFunction:
;do stuff
call trampoline ;calls the original function
;do stuff
ret

trampoline:
push ebp
mov ebp, esp
sub esp, 0x40
jmp trampolinetarget

If something else tries to hook the original function, they'll just overwrite that jmp instruction, and their trampoline will call your function. Microsoft's Detours library can handle all of this for you.

Answer 2

Intel PIN in Probe Mode can do just. You can inject C code and PIN will even handle the context switching for you.