Synchronize promises created asynchronously

Question

I'm working on a little nodejs program and have some troubles to know when all async operations of the program are completed.

For now, the program do the following steps:

1/ Spawn a process with some parameters. This process will print data on its stdout.

2/ Listen to the process stdout "data" event and everytime something is printed the program calls a function (which we will call "process"), using the data.

3/ This process function will eventually insert the data into a mongo database and sending a message to a amqp server.

3/ When there is no more data, the program is idling because the connection to the amqp and mongo database are still alive, so I need to know when all the work is done to be able to close the connections.

So, I tried to use when.js to make use of promises, but I can't make it work for what I'm trying to achieve.

I made the "process" function return a promise, which will be resolved when mongodb insert and amqp message sending are done. In my program, I'm creating an array which will receive all the promise to be able to call when.all() to know when they are all resolved.

But since I'm creating promises asynchronously when the spawned process is printing data to its stdout stream, the call to when.all() is made with an empty array which seem to be resolved immediately.

Here is a code sample which illustrate what I'm achieving:

var when = require('when')
  , _    = require('lodash')
  , cp   = require('child_process');

var promises = [];

function process(data) {
    var deferred = when.defer();

    setTimeout(function () {
        deferred.resolve(true);
    }, 3000); // Let's say we need 3 seconds to process and save data

    return deferred.promise;
}

var ls = cp.spawn('ls', ['-la', '/usr/bin']);
ls.stdout.on('data', function (data) {
    console.log('Received data, creating a promise to notify when this data is processed.');
    promises.push(process(data));
});

when.all(promises).then(function (values) {
    console.log('All promises are now resolved', values);
});

As you might have guessed, the output of this program is:

All promises are now resolved []
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.
Received data, creating a promise to notify when this data is processed.

Is there any way to make this code sample print console messages in the expected order (which is having the first line printed last) ?

Thanks.

Answer 1

You need to call when only after all promises have been made. In your comment you mentioned:

I guess I cannot know when everything is done this way

Which is incorrect. You can know when everything is done using the close event:

ls.on('close',function(){
    when.all(promises).then(function (values) {
        console.log('All promises are now resolved', values);
    });
});