Should I divide C++ program into multiple interacting applications?

Question

We are discussing proposal to split a big C++ program into multiple separate executables that would communicated using shared memory. The shared data structures are large, so we do not want to use loopback network or any other approach that would just copy them.

The arguments for splitting are that every part can be developed separately, potentially replacing it with the alternative implementation, even in another language. It would naturally prevent accessing private data and code and the processes would obviously run in separate threads.

The arguments against would be that C++ has built-in means to structure also a large and complex project, hiding data and functions as designed. It is possible use C++ multithreading to employ all cores of the CPU. In this case the data can be passed by reference from module to module without tricks.

Is there a known widely accepted view about dividing a C++ program into multiple binaries running in parallel on the same host? Do any widely known programs work this way?

Suggestions to implement in another language are outside the scope of this question.

Answer 1

multiple separate executables that would communicated using shared memory

This is a fairly bad idea, because suddenly your program is impossible to debug. If process A crashes while looking at some data in the shared area, you can pause it in the debugger - but process B is free to carry on and overwrite the suspect data. Shared-memory concurrency is generally considered troublesome even when the accessors are part of the same multi-threaded program.

I think the only big example I can think of that uses shared memory is X windows.

Cache coherency issues mean that shared memory may have surprising semantics; normally you'd expect writing to address A then address B would imply that another process which sees the updated address B value would also see the address A change, but that may not always be true.

But it sounds like the application isn't even multithreaded yet. I would suggest starting with this as it's considerably easier to get working and the primitives are better.

every part can be developed separately, potentially replacing it with the alternative implementation, even in another language

You have to be very rigorous about defining the layout of the shared memory if you're going to do this. It becomes a lot more like an in-memory database or file format. You can't, for example, have any objects in the shared memory with virtual functions on, they all have to be POD types.

Answer 2

The arguments for splitting are that every part can be developed separately,

You don't need separate executables for doing that. You don't even need separate binaries (dll or so) for that. You just need well defined interfaces.

potentially replacing it with the alternative implementation,

Again, this does not require a separate executable.

even in another language.

Do you have a legitimate business case that requires this? If not, don't even consider it.

Answer 3

tl;dr - it doesn't seem like the possible benefits of multiple processes are things you care about, and the things you do mention as concerns work at least as well with multiple threads.

---

Depending on platform, there isn't necessarily a huge distinction between multiple-process shared memory, and single process multiple threads.

In both cases you have multiple schedulable threads of execution sharing some resources, and roughly the same requirements for synchronizing access to them.

The main differences are:

1. multiple processes are potentially harder to debug as pjc50 says (you can't just stop the whole thing in a debugger, or get a single core file)

2. multiple processes are potentially easier to test, as you have a natural boundary for mocking other components

   Obviously you can test well-defined components in-process, but that will be a unit test process rather than a standalone production process. This distinction probably doesn't matter in most situations.

3. you can't use raw pointers in memory shared between processes, as it's mapped into a different address space. You can use offsets or some other type of handle or index instead though, and if you don't use pointers it makes no difference
4. you can really only share POD or trivially-initializable types in shared memory - vtable pointers have the same problem as any other (RTTI and function addresses are also process-specific)

Now, the benefits people usually look for in multiple processes are:

1. resilience - a fatal error in one process doesn't take everything down
2. security - running untrusted plugins or scripts without giving access to your entire address space, or under a less-privileged user
3. asynchronicity - this is really only a concern if you want something to run in parallel but the rest of your program can't be changed to accomodate that

4. resource partitioning - some programs (eg. web servers) can use all the file descriptors allowed to a single process and must run multiple instances.

   32-bit programs could have a similar problem with address space, but that's not likely to be a limitation if you build 64-bit. You might want one process per socket on a NUMA system, but then you're deliberately not sharing memory between them, so it doesn't match your case anyway.

Note that none of the potential benefits of using multiple processes require sharing memory. Resilience and security in particular work best with serialized communication.