How to pipe one readable stream into two writable streams at once in Node.js?

Question

The goal is to:

1. Create a file read stream.
2. Pipe it to gzip (zlib.createGzip())

3. Then pipe the read stream of zlib output to:

   1) HTTP response object

   2) and writable file stream to save the gzipped output.

Now I can do down to 3.1:

var gzip = zlib.createGzip(),
    sourceFileStream = fs.createReadStream(sourceFilePath),
    targetFileStream = fs.createWriteStream(targetFilePath);

response.setHeader('Content-Encoding', 'gzip');

sourceFileStream.pipe(gzip).pipe(response);

... which works fine, but I need to also save the gzipped data to a file so that I don't need to regzip every time and be able to directly stream the gzipped data as a response.

So how do I pipe one readable stream into two writable streams at once in Node?

Would sourceFileStream.pipe(gzip).pipe(response).pipe(targetFileStream); work in Node 0.8.x?

Answer 1

I found that zlib returns a readable stream which can be later piped into multiple other streams. So I did the following to solve the above problem:

var sourceFileStream = fs.createReadStream(sourceFile);
// Even though we could chain like
// sourceFileStream.pipe(zlib.createGzip()).pipe(response);
// we need a stream with a gzipped data to pipe to two
// other streams.
var gzip = sourceFileStream.pipe(zlib.createGzip());

// This will pipe the gzipped data to response object
// and automatically close the response object.
gzip.pipe(response);

// Then I can pipe the gzipped data to a file.
gzip.pipe(fs.createWriteStream(targetFilePath));

Answer 2

Pipe chaining/splitting doesn't work like you're trying to do here, sending the first to two different subsequent steps:

sourceFileStream.pipe(gzip).pipe(response);

However, you can pipe the same readable stream into two writeable streams, eg:

var fs = require('fs');

var source = fs.createReadStream('source.txt');
var dest1 = fs.createWriteStream('dest1.txt');
var dest2 = fs.createWriteStream('dest2.txt');

source.pipe(dest1);
source.pipe(dest2);

Answer 3

you can use "readable-stream-clone" package

const fs = require("fs");
const ReadableStreamClone = require("readable-stream-clone");

const readStream = fs.createReadStream('text.txt');

const readStream1 = new ReadableStreamClone(readStream);
const readStream2 = new ReadableStreamClone(readStream);

const writeStream1 = fs.createWriteStream('sample1.txt');
const writeStream2 = fs.createWriteStream('sample2.txt');

readStream1.pipe(writeStream1)
readStream2.pipe(writeStream2)