Limiting an ATL exe server to one process (instance) only

Question

I have an ATL exe server written with C++ (Visual Studio 2010, with merging the proxy-stud dll). The executable is compiled for Windows 7 x86 & x64. The following happens on both architectures:

The ATL exe server should function as a "server process", that is - one process (MyATLServer.exe, ONLY one!) should exists per machine and many clients (let's keep it simple: on the same machine) are consuming COM objects from it. The server saves application state (in memory) and all clients should "share" this sate, by using the server's exposed COM objects.

The exe server usually starts when the system starts because it is invoked by a COM call to create one of it's hosted objects. The call originates from within the spoolsv.exe process (Print spooler service). This causes the server process to run under the 'SYSTEM' user (I guess due to the fact that spoolsv.exe runs with 'SYSTEM').

When one of the clients creates a COM object from the ATL server, another process (MyATLServer.exe again) is instantiated (running under the user who's logged in to Windows), thus, it cannot share application state with the "original" one (which is running under 'SYSTEM'). A second client will connect to the one instantiated by the logged in user (the "second one").

After searching countless forums on the WEB I've managed to get to the following conclusions:

1) My ATL server uses the default (auto-generated by VS2010) ATL-Module which inherits from ATL::CAtlExeModuleT<>. Digging into the ATL headers I'm positive that this module calls AtlComModuleRegisterClassObjects with the correct flags for this kind of use (dwClsContext = CLSCTX_LOCAL_SERVER, flags = REGCLS_MULTIPLEUSE | REGCLS_SUSPENDED). So it rules this one out.

2) Using start -> run -> dcomcnfg -> DCOM Config -> MyATLServer -> Properties I've set the "This user" option under the 'Identity' tab (with the local administrator user). In the 'Location' tab, the option 'Run application on this computer' is not selected and grayed-out. This caused me to search again and I got here: http://social.technet.microsoft.com/Forums/en-US/w7itprosecurity/thread/4f63ee11-e472-40f9-85db-a6b235d7579c. The link explains that the x64 version of the dcomcnfg utility is buggy and one should use the 32-bit version of it on x64 Windows (start->run->'mmc comexp.msc /32'). After checking it on two system (one of each) I found out that the 'Run application on this computer' option is grayed out and not selected on both.

From this point I'm totally lost (and trying to suppress the frustration... :-)). Am I on the correct path? Did anyone do this before?

Or the original intent - How do I make my ATL exe server "singled-processed"?

Thanks!

Omri

Answer 1

Ok... So finally I'm going to answer myself :-)

I've found some time to try and solve this issue again (and again...). After setting 2 different new dev machines (Win7x86 & Win7x64) and using ONLY the 32bit version of dcomcnfg - one machine worked, the 32bit one (Win7x86)!

Digging into the "non-working" machines, I found out that I've encountered GUIDs hell! On the x64 dev machine, the COM objects were registered with different GUIDs than the values in the code files (the generate code). To make it even difficult, I've used connection points, which were also registered with invalid GUIDs. By that I mean - The GUID values in the registry were not correct! Until now, I have no idea why this happens, but I'm 100% sure that the COM wizards in VS2010 are buggy (generate partial code, if any...), maybe the code these wizards generate are causing registration problems.

To solve the issue I've manually deleted all relevant registry keys of the ATL server, canceled the auto-COM registration in the VS project and registered the EXE myself (on x64 using the %WINDIR%\SysWOW64\cmd.exe console, not the native x64 one).

After registering the server manually, all worked fine (as expected, a single process, shared among several clients) and there were no grayed check boxes in the dcomcnfg configuration utility.

I'm sure this solution has a lot of unexplained gaps, but it work for me (for now...).

Cheers ;-)