Overriding Node.js HTTP parser

Question

I am using Node's basic http.request() function without problem on normal HTTP servers. I need to use http.request() (or similar) with SHOUTcast servers. The SHOUTcast "protocol" is fully compatible with HTTP, except for one detail... the first response line.

Normal HTTP servers respond with:

HTTP/1.1 200 OK

SHOUTcast servers respond with:

ICY 200 OK

Again, the rest of the protocol is the same. The only difference is HTTP/1.x vs. ICY.

I would like to extend, subclass, or somehow modify Node's http.request() function so that I can make it work with SHOUTcast servers. Connecting to SHOUTcast with Node has been done before, but only by re-inventing the entire wheel. I'd rather not do that for what amounts to be a minor protocol difference.

My Question: Is there a way to do one of the following?

1. Extend or override the relevant parts of Node's HTTP parser. (I doubt this is possible, as it seems the parser is native code.)

2. Create my own bit of code that parses the relevant parts of HTTP, but reuses as many existing Node components for HTTP as possible.

3. Create a simple internal proxy (or somehow relay data) so that I can modify that first server response line prior to it reaching Node's HTTP parser.

4. Something else?

I have also considered using Shred, but it does not offer an option to stream the response. (It waits for the entire server response to be complete before firing an event, which won't work for streaming servers where the data can run indefinitely.) Along those same lines, I tried Request, but it uses Node's own HTTP parser, so I get the same parse errors as I would with the native HTTP client.

Answer 1

I've come up with another way to do this, similar to the internal proxy, but without the extra connection. It seems that it is possible to override the internally used socket for the HTTP client. When this is done, it is easy to hook in and modify data before passing it off to the original internal socket ondata function.

Usage

var httpicy = new HttpIcyClient();
httpicy.request(/* your normal request parameters here */);

Source

var http = require('http');

var HttpIcyClient = function () {};
HttpIcyClient.prototype.request = function (options, callback) {
    var req = http.request(options, callback),
        originalOnDataFunction,
        receiveBuffer = new Buffer(0);

    req.on('socket', function (socket) {
        originalOnDataFunction = socket.ondata;
        socket.ondata = function (d, start, end) {
            receiveBuffer = Buffer.concat([receiveBuffer, d.slice(start, end)]);
            if (receiveBuffer.length >= 4) {
                socket.ondata = originalOnDataFunction;
                if (receiveBuffer.toString('ascii', 0, 4) === 'ICY ') {
                    receiveBuffer = Buffer.concat([new Buffer('HTTP/1.0 ', 'ascii'), receiveBuffer.slice(4)]);
                }
                socket.ondata.apply(this, [receiveBuffer, 0, receiveBuffer.length]);
            }
        };
    });
    return req;
}

Answer 2

3 . Create a simple internal proxy

Bang. You got it.

Use the Node.js net object to create a TCP server that actually sits between the user and the http server. Replace the first line of all requests and responses as you've described, and you can continue to use http server essentially as-is. (You create an internal http server on a non-standard port, the net server on another port, then chain up streams between the two, with a bit of code to intercept the first chunk of data from the requesting user, and the first chunk of data from the responding server.)

All of your code in Javascript and it's all streaming with (almost) no bufferbloat.

EDIT, it seems I misread your post slightly. I see that you're not trying to implement a Node.js shoutcast server, but trying to access data from it. The net object should still work, but not quite in the same way.

You'll need to use net.connect to talk to the specified shoutcast server, and use a new net server that knows about the net.connect stream to proxy between it and your http.request(). So, it would work like:

+---------------+    +----------+    +-----------+    +----------------+
| http.request()|--->|net.Server|--->|net.connect|--->|SHOUTcast Server|
|               |<---|          |<---|           |<---|                |
+---------------+    +----------+    +-----------+    +----------------+
                  TCP             JS               TCP

That's about how I'd structure it.