Okay so not too long ago i was introduced to the concept of hooking/detouring, it took me some time to get my head around, but despite the lack of information available, eventually i was able to hook BeginScene and EndScene using IDA, MS Detours and one of the sample d3d9 applications for testing.
But i realised, that was just one way to do it, so i decided that i'd like to also become familiar with the other methods i'd seen used.
This led me to do some studying on V-Tables, and more specifically, a practice that has been described as dynamic patching of V-Tables.
I found that the typical way of doing this for DirectX is to create a "Dummy device" Then alter it's V-Table, which supposedly affects all instances of the device class.
I went on to try create a scaled down environment using my own class with a single virtual function and some code that instantiated said class, then continuously called said function.
I then injected some code that had access to a copy of this class, it created a new instance and attempted to patch the V-Table function pointer with that of another function.
But i soon found that the way i'd set it up just didn't work as i'd expected.
I'm quite certain it modified the V-Table, but only for that instance of the class.
(i'd tested this by combining the code into a single program which simply called the function, grabbed the V-Table, modified it to point to another function, then called the function again to confirm it had been swapped)
So i wonder, what is it about DirectX that makes it possible to patch the original V-Table using a new instance of a device class/object?
Is hot patching something specific to COM, or is there some other explanation, maybe i have it all wrong?
(EDIT)
Here's an example of what i'm trying to achieve, but i'd like to move the code that grabs the V-Table and modifies the function pointer, and put it into a DLL.
using namespace std;
class A {
public:
virtual void doThing() {
cout << "Class A doThing" << endl;
}
};
void dynamicOverride() {
cout << "Function replaced" << endl;
}
int main()
{
A* a = new A();
a->doThing();
void** vtable = *(void***)a;
DWORD curProtection;
VirtualProtect(&vtable[0], 4, PAGE_EXECUTE_READWRITE, &curProtection);
vtable[0] = dynamicOverride;
a->doThing();
system("pause");
return 0;
}
I've seen this done in code that hooks various d3d9 functions, most refer to it as creating a dummy device.
But in this case when i try use a class in a DLL and inject the code to modify the V-Table for that class. It never affects the instance of that class in the target process.
