What we have here is a genuine bug. I filed: https://github.com/mozilla/pdf.js/issues/3634
Since the data transfer does not specify a content-length, but uses chunked transfer encoding, pdf.js will use an initial buffer of 64kb that is doubled each time it would overflow. However, once the transfer is complete pdf.js will not shrink that buffer to the actual size, nor will it remember the actual size, so that upon download the whole over-sized buffer (still the initial 64kb in your example) will be transferred.
I don't think there is a real work-around, short of not using pdf.js at all (which is a user choice).