What's the limit of spawning child_processes?

Question

I have to serve a calculation via algorithm, I've been advised to use a child process per each opened socket, what I am about to do is something like that:

var spawn = require('child_process').spawn;
var child = spawn('node', ['algorithem.js']);

I know how to send argument to the algorithm process and how to receive results.

What I am concerned about, is how many socket (each socket will spawn a process) I can have? How can I resolve this with my cloud hosting provider? so that my app gets auto scaled?

What's the recommended node js cloud hosting provider?

Finally, is this a good approach in using child processes?

Answer 1

Yes, this is a fair approach when you have to do some heavy processing in node. However, starting a new process introduces some overhead, so be aware. The number of sockets (file descriptors) you can open is limited by your operating system. On Linux, the limits can seen using for example the ulimit-utility.

One alternative approach, that would remove the number of sockets/processes worry, is to run a separate algorithm/computation-server. This server could spawn N worker threads and would listen on a socket. When a computation request is received, this can for example be queued and processed by the first available thread. An advantage of this approach is that your computation server can run on any machine, freeing up resources for your node instance.