Ruby Sinatra with multi part post request

Question

Not sure whether this is a problem caused by multi part post request. From client side, I sent a post request using node.js. The post body is a JSON array

var post_req = http.request(options, function(res) {
    res.setEncoding('utf8');
    res.on('data', function (chunk) {
      console.log('Response: ' + chunk);
    });
  }).on('error',function(){});
post_req.write(data.toString().split("\0").join(""));
post_req.end();

On server side(ruby sinatra), I have a post handler, which just prints out the JSON array

post '/prediction'  do
puts request.body.read
end

It prints out a strange incomplete json array with a strange string "13b2b" before the array. The received data can be found here: https://gist.github.com/wenchaojiang/9880181

I am sure the data i sent is a valid JSON array, why sinatra can not receive it completely and where does the strange "13b2b" come from? any idea?

-----------Edit -------------

The sample data is huge. it can be found here: https://gist.github.com/wenchaojiang/9881235

Another strange thing is, when I use following node.js server, the json can be received correctly

http.createServer(function (req, res) {
if (req.method == "POST") {
    var queryData = "";
    req.on('data', function(data) {
        queryData += data;
    });

    req.on('end', function() {
       var obj = JSON.parse(queryData);

       console.log(JSON.parse(queryData).payload);

       res.writeHead(200, "OK", {'Content-Type': 'text/plain'});
       res.end();
    });
}   
}).listen(3001, '0.0.0.0');

-----------------Edit ----------------------------

It also seems that the Sinatra will receive 2 post requests even if I only sent it once from the node.js side. The server the received incomplete json mentioned above from the first request and the second post request leads to the following error

!! Invalid request
Invalid HTTP format, parsing fails.
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/thin-1.5.0/lib/thin/request.rb:82:in `execute'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/thin-1.5.0/lib/thin/request.rb:82:in `parse'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/thin-1.5.0/lib/thin/connection.rb:39:in `receive_data'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/eventmachine-1.0.0/lib/eventmachine.rb:187:in `run_machine'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/eventmachine-1.0.0/lib/eventmachine.rb:187:in `run'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/thin-1.5.0/lib/thin/backends/base.rb:63:in `start'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/thin-1.5.0/lib/thin/server.rb:159:in `start'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/rack-1.4.1/lib/rack/handler/thin.rb:13:in `run'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/rack-1.4.1/lib/rack/server.rb:265:in `start'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/rack-1.4.1/lib/rack/server.rb:137:in `start'
/usr/local/rvm/gems/ruby-1.9.3-p194/gems/rack-1.4.1/bin/rackup:4:in `<top (required)>'
/usr/local/rvm/gems/ruby-1.9.3-p194/bin/rackup:19:in `load'
/usr/local/rvm/gems/ruby-1.9.3-p194/bin/rackup:19:in `<main>'

Answer 1

I think (although I can’t reproduce exactly what you’re seeing) this is due to the Thin server not handling chunked transfer encoded requests. Node is breaking up the file into several parts and trying to send each chunk separately. The strange string (13b2b) is the hex value of the size of the first chunk (although that doesn’t actually match the data in your Gist) and it is cut off because the client is expecting to send the rest later.

The Node server works because it properly handles chunked requests, and the other (malformed) requests that Sinatra is seeing are the following chunks.

You can try using another server in your Sinatra app to see if it works. Webrick seems to be okay:

set :server, 'webrick'