I have a file which consists of 69-byte messages. No EOL characters- just message after message. The total number of bytes in the file is exactly 11,465,930,307, which is (11,465,930,307/69) = 166,172,903 messages.
My program memory-maps the file in to a byte array, looks at each 69-byte message and extracts the timestamp. I keep track of which message number I am on and then the timestamp and the message number go in a RowDetails
object, which goes in a std::vector<RowDetails>
called to_sort
, so that I can effectively sort the whole file by timestamp.
std::cout << "Sorting....." << to_sort.size() << " rows..." << std::endl;
std::sort(std::begin(to_sort), std::end(to_sort));
However, then I create a new file which is sorted:
unsigned long long total_bytes=0;
unsigned long long total_rows=0;
ofstream a_file("D:\\sorted_all");
std::cout << "Outputting " << to_sort.size() << " rows..." << std::endl;
std::cout << "Outputting " << (to_sort.size()*69) << " bytes..." << std::endl;
for(RowDetails rd : to_sort){
for(unsigned long long i = rd.msg_number*69; i<(rd.msg_number*69)+69; i++){
a_file << current_bytes[i];
total_bytes++;
}
total_rows++;
}
std::cout << "Vector rows: "<< total_rows <<std::endl;
std::cout << "Bytes: " << total_bytes <<std::endl;
My output:
No. of total bytes (before memory-mapping file): 11,465,930,307 CORRECT
Sorting....... 166,172,903 rows CORRECT
Outputting 166,172,903 rows.... CORRECT
Outputting 11,465,930,307 bytes CORRECT
Vector rows: 166,172,903 CORRECT
Bytes: 11,465,930,169 ERROR, THIS SHOULD BE 307, not 169
How can I process the correct number of rows, but my counter, counting total bytes is wrong??
When looking at the output file in Windows 7 explorer it says size: 11,503,248,366 bytes, even though the original input file (which I memory-mapped) said the correct 11,465,930,307.