Injecting a CLR host into a running process - possible?

Question

This way lies madness, I know.

I have an executable (the source code to which I do not have access) that I would like to extend via .NET. It's a native executable, so I would need to inject a CLR host in order to do this. My basic idea is to provide scripting-like functionality via a .NET language (C#, for example) and provide hooks in the target executable for the scripts to manipulate, and vice versa.

I know I'll need to use various techniques to achieve this - DLL injection, some runtime ASM injection, etc., but what I'd like to know is this: is what I'm talking about possible? Better still - has anyone done something like this before?

Answer 1

You can do this using COM interop, if your native executable is capable of using COM objects. If you register your .NET assembly for interop then your native executable can use your .NET classes just like "ordinary" COM objects, and when the first one is created it will spin up the CLR inside the native process. Similary if you can expose COM objects from the native executable then you can use them from your .NET code if you create an interop assembly (or even without a type library, if you use only IDispatch).

The basics are straightforward but I'm just scratching the surface - for a serious project like this you need a serious reference. I highly recommend .NET and COM, The Complete Interoperability Guide by Adam Nathan. It's a big book that doesn't leave much unsaid, and it also has a lot of great information on designing .NET and COM classes to interop cleanly. It also explains how to host the CLR directly inside a native app, but that option may not be practical without access to the source code. I would definitely start with the COM interop route, and only host the CLR natively if you have no other option.

Answer 2

We actually do something very similar for an automation framework. You might want to take a look at the EasyHook project on CodePlex. Its got a ton of nice features including a built in IPC Communicator for cross-process communication. It required a bit of leg work but should do exactly what you are looking for.