Superagent moves response callback position in async waterfall

Question

I have a simple working superagent/async waterfall request that looks like this:

  request = require 'superagent'
  user = request.agent()
  async.waterfall [
    (cb)->
      user.post('http://localhost:3000/form').send(name: 'Bob').end(cb)
  ], (err, res)->
    console.log err
    console.log res

This successfully prints my full http response, and err is undefined.

If I perform the exact same thing with an extra step:

  request = require 'superagent'
  user = request.agent()
  async.waterfall [
    (cb)->
      user.post('http://localhost:3000/form').send(name: 'Bob').end(cb)
    (err, res)->
      # this is never reached
      cb()
  ], (err, res)->
    console.log err # this now prints out the response
    console.log res # this is undefined

err is now the response. res is undefined. Is this a superagent issue I'm running into here, or am I simply using async's waterfall incorrectly?

Answer 1

It is a SuperAgent "issue" in how they've chosen to handle the function that is passed as the callback. If that function is expecting exactly two arguments as reported by the length property, then the "traditional" err and res are given like Async would like. If the function you pass does not report its length to be two, then the first argument given is res. Here's SuperAgent's source for handling callbacks:

Request.prototype.callback = function(err, res){
  var fn = this._callback;
  if (2 == fn.length) return fn(err, res);
  if (err) return this.emit('error', err);
  fn(res);
};

To guarantee that your callbacks are invoked as intended, I would suggest passing an anonymous function to end so that it definitely reports its length as two so you can get any errors passed to your callback.

request = require 'superagent'
user = request.agent()
async.waterfall [
  (cb) ->
    user.post('http://localhost:3000/form').send(name: 'Bob').end (err, res) ->
      cb err, res
  (err, res) ->
    # err should be undefined if the request is successful
    # res should be the response
    cb null, res
], (err, res) ->
  console.log err # this should now be null
  console.log res # this should now be the response

Answer 2

Async waterfall passes error directly to it's callback. The second function in array receives only one argument - res. And every function in array should have it's own callback as the last argument. If error happens you should catch in in waterfall's callback. Try:

async.waterfall([
  function(cb){
    superagent...end(cb);
  },
  function(res, cb){ //Note cb here.
    //If you don't pass res here, waterfall's callback will not receive it
    cb(null, res); 
  }], function(err, res){
    //If superagent fails you should catch err here
    should.not.exist(err);
    //res is defined because you passed it to callback of the second function
    should.exist(res); 
 });