Long-running computations in node.js

Question

I'm writing a game server in node.js, and some operations involve heavy computation on part of the server. I don't want to stop accepting connections while I run those computations -- how can I run them in the background when node.js doesn't support threads?

Answer 1

I can't vouch for either of these, personally, but if you're hell-bent on doing the work in-process, there have been a couple of independent implementations of the WebWorkers API for node, as listed on the node modules page:

• http://github.com/cramforce/node-worker
• http://github.com/pgriess/node-webworker

At first glance, the second looks more mature, and these would both allow you to essentially do threaded programming, but it's basically actor-model, so it's all done with message passing, and you can't have shared data structures or anything.

Also, for what it's worth, the node.js team intends to implement precisely this API natively, eventually, so these tools, even if they're not perfect, may be a decent stopgap.

Answer 2

var spawn = require('child_process').spawn;
listorwhatev = spawn('ls', ['-lh', '/usr']);//or whatever server action you need

//then you can attach events to that list like this
listorwhatev.on('exit', function(code){});

///or in this ls example as it streams info
listorwhatev.stdout.on('data', function(info){sys.puts(info);});

ensure spawn process occurs once per application then feed stuff into it and watch it for events per connection. you should also check that the listorwhatev is still running before handling it. As we all love those uncaught errors in node crashing the app don't we ;) When the spawn (pid) is exited though a kill or something bad happens on your machine and you didn't exit the spawn in your code gracefully, your stream event handler will crash your app.

Answer 3

and some operations involve heavy computation on part of the server

How did you write code that is computation heavy in the first place. That's very hard to do in node.js.

how can I run them in the background when node.js doesn't support threads

you could spawn a couple of worker(node) instances and communicate with accepting connections(node instance) using for example redis blocking pop. Node.js redis library is none blocking.