COM does not support this activation scenario. It is supposed to be covered by an in-process server, do make sure that isn't the way you want to do it given its rather major advantages.
Using REGCLS_SINGLEUSE is the alternative, but this requires you extending your object model to avoid the storm of server instances you now create. The Application
coclass is the boilerplate approach. Provide it with factory methods that gives you instances to your existing interfaces.
I'll mention a drastically different approach, one I used when I wanted to solve the same problem as well but required an out-of-process server to take advantage of bridging a bitness gap. You are not stuck with COM launching the server process for you, a client can start it as well. Provided of course that it knows enough about the server's installation location. Now a client of course has complete control over the server instance. The server called CoRegisterClassObject() with an altered CLSID, I xored part of the guid with the process ID. The client did the same so it always connected with the correct server. Extra code was required in the client to ensure it waits long enough to give the server a chance to register its object factories. Worked well.