Using async.waterfall

Question

I'm using node.js and the async package.

Here's the code I have:

async.waterfall(
[
    function(callback) {
        var data = getSomeData();
        callback(null, data);
    },
    function(data, callback) {
        someFunctionThatNeedsData(data);
        callback(null, 'done');
    }
],
function(err, result) {
}
);

getSomeData has an asynchronous HTTP request that grabs some data from a web service. I'd like to wait until I get a response, and then return that data and pass it to someFunctionThatNeedsData.

What I expected was that getSomeData -- including the callback inside of it -- would have to complete before moving on to invoke someFunctionThatNeedsData.

The problem is that, despite using the waterfall function here, data is undefined by the time it gets to someFunctionThatNeedsData.

Additionally, from console.log I can see that the end of getSomeData is reached before the callback inside of getSomeData even begins.

Am I using waterfall incorrectly, or is it just not the right tool here? If it's just not right, what can I use to achieve the desired effect?

Or do I have to resign to having deeply nested callbacks (which, with future work, I will) and have to just mitigate it by extracting inline code into named functions?

Answer 1

getSomeData() has an asynchronous http request that grabs some data from a web service.

This is the issue. The execution flow already continued to the callback and executed it. This is how asynchronous functions work!

You have to pass the callback to getSomeData, which calls it once the HTTP request finished. So yes: You may need to nest the callbacks.

Answer 2

If you have async operation. You don't necessary to use async.waterfall. You could just do that in a promise chain style.

getSomeData().then(function(data) 
{
    var changeData = changeYourData(data);
    return changeData;
}).then(function(changedData)
{
    // some more stuff with it. You can keep on forwarding to the next `then` 
}).catch(function(err)
{
    // if any error throw at any point will get catch here
}).finally(function() 
{
    // this one will guarantee get call no matter what,
    // exactly the same like async.waterfall end of chain callback
});

This example will work with Q, When, and any promise lib that follow standard.

If you need to use async.waterfall (because you could drive it with an Array.map) You just need to callback in your then

async.waterfall(
[
    function(callback) {
        // A 
        getSomeData().then(function(data)
        {
            callback(null, data);
        });
        // B - just throw the whole thing in
        callback(null , getSomeData());
    },
    function(data, callback) {
        // A
        someFunctionThatNeedsData(data);
        // B
        data.then(function(resolvedData)
        {
             someFunctionThatNeedsData(resolvedData);
             callback(null, 'done');
        });
    }
],
function(err, result) {
});

Hope this help.