Draw DirectX/OpenGL Graphics on an existing graphics application

Question

First off, let me just apologize right off the bat in case this is already answered, because I might just be searching it under irregular search terms.

I am looking to draw 2D graphics in an application that uses DirectX to draw its own graphics (A game). I will be doing that by injecting a DLL into the application (that part I have no questions about, I can do that), and drawing my graphics. But not being really good at DirectX/OpenGL, I have a couple of fundamental questions to ask.

1) In order to draw graphics on that window, will I need to get a pre-existing context from the process memory, some sort of handle to the drawing scene?

2) If the application uses DirectX, can I use OpenGL graphics on it?

Please let me know as to how I can approach this. Any details will be appreciated :-) Thank you in advance.

Answer 1

"1) In order to draw graphics on that window, will I need to get a pre-existing context from the process memory, some sort of handle to the drawing scene?"

Yes, you need to make sure your hooks catch the important context creation functions. For example, all variations of CreateDevice in d3d are interesting to you.

You didn't mention which DirectX you are using, but there are some differences between the versions. For example, At DirectX 9 you'd be mostly interested in functions that: 1. Create/return IDirect3DSwapChain9 objects 2. Create/return IDirect3DDevice9,IDirect3DDevice9Ex objects

In newer versions of DirectX their code was splitted into (mostly) Device, DeviceContext, & DXGI. If you are on a "specific mission" share which directx version you are addressing.

Apart from catching all the needed objects to allow your own rendering, you also want to catch all presentation events ("SwapBuffers" in GL, "Present" in DX), Because that's time that you want to add your overlay.

Since it seems that you are attempting to render an overlay on top of DX applications, allow me to warn you that making a truly generic solution (that works on all games) isn't easy. mostly due to need to support different DX versions along with numerous ways to create If you are focused on a specific game/application it is, naturally, much easier.

"2. If the application uses DirectX, can I use OpenGL graphics on it?"

Well, first of all yes. It's possible. The terminology that you want to search for is OpenGL DirectX interoperability (or in short interop)

Here's an example: https://sites.google.com/site/snippetsanddriblits/OpenglDxInterop I don't know if the extension they used is only available in nVidia devices or not - check it.

Another thing about this is that you need a really good motivation in order to do it, generally I would simply stick with DX for both hooking and rendering. I assume that internal interop between different DX version is better option. I'd personally probably go with DirectX9 for your own rendering code. Of course, if you only need to support a single DirectX version, no interop needed.

Bonus: If you ever need to generate full wrappers of C++ classes, a quick n' dirty dll wrapper, or just general global function hook, feel free to use this lib that i created: http://code.google.com/p/hookit/ It's far from a fully tested tool, just something i hacked 2 days, but I found it super useful.

Note that in your case, i recommend just to use VTable hooking, you'll probably have to hardcode the function offset into the table, but that's not likely to change.

Good luck :)

Answer 2

Your approach in injecting an DLL is indeed the right way to go. Programs like FRAPS use the same approach. I can't tell you about the method for Direct3D, but for OpenGL you'd do about the following things:

First you must Hook into the functions wglMakeCurrent, glFinish and wglSwapBuffers of opengl32.dll so that your DLL notices when a OpenGL context is selected for drawing. Pass their calls through to the OS. When wglMakeCurrent is called use the function GetPixelFormat to find out if the window is double buffered or not. Also use the glGet… OpenGL calls to find out which version of OpenGL context you're dealing with. In case you have a legacy OpenGL context you must use different methods for drawing your overlay, than for a modern OpenGL-3 or later core context.

In case of a double buffered window use your Hook on wglSwapBuffers to perform further OpenGL drawing operations. OpenGL is just pens and brushes (in form of points, lines and triangles) drawing on a canvas. Then pass through the wglSawpBuffers call to make everything visible.

In case of a single buffered context instead of wglSwapBuffers the function to hook is glFinish.

Draw 2D with OpenGL is as simple as disable depth buffering and using an orthographic projection matrix. You can change OpenGL state whenever you desire to do so. Just make sure you restore everything into its original condition before you leave the hooks.