Different results when populating a vector from a stream and a buffer

Question

I have a binary file that I am loading into a byte vector, and I'm unexpectedly getting different results depending on how I read the file. I suspect that it has something to do with my use of istream_iterator, but I would like to better understand what is happening. By way of background, the data being loaded is simply a byte stream. If it matters, and I think it may, the final dozen or so bytes are 0x00.

Case 1

ifstream is("file.dat", ifstream::binary);
vector<uint8_t> v;
v.assign(istream_iterator<uint8_t>(is), istream_iterator<uint8_t>());

Case 2

ifstream is("file.dat", ifstream::binary);
vector<uint8_t> v;
is.seekg(0, in.end);
int length = is.tellg();
is.seekg(0, in.beg);
char* buffer = new char[length];
is.read(buffer, length);
v.assign(buffer, buffer + length);

Case 1 results in a smaller vector. The initial data in both vectors are the same, but in Case 1 it is truncated.

Answer 1

Vlad has provided an improved version of Code #1, I'll provide a better Code #2:

basic_filebuf<uint8_t> ifb;
ifb.open("file.dat", ios_base::in | ios_base::binary);

vector<uint8_t> v;
v.resize(ifb.pubseekoff(0, ios_base::end));
ifb.pubseekpos(0);
ifb.sgetn(&v[0], v.size());

The fact is that an ifstream is not intended for processing unformatted data, even when ios_base::binary is used to open it. That flag is somewhat misnamed -- it disables newline translations, but it doesn't cause insertion and extraction operations to use binary encodings.

This version should be quite a lot faster, not only because there's less copying when the vector is preallocated, but because filebuf xsgetn can be expected to transfer large blocks, whereas iterators access single elements. Even if there is a buffer used to do large block transfers at the I/O level, the virtual calls per-character imposed by the iterators will kill performance.

Answer 2

Try the following

ifstream is("file.dat", ifstream::binary);
vector<uint8_t> v;
v.assign(istreambuf_iterator<char>(is), istreambuf_iterator<char>());

As for istream_iterator then it uses operator >>.