C++ string too big, trailing characters truncated

Question

I am making a program that when started, will write a predefined string into a file. The file is about 5 mb in size so filling the string with 5 mb of data in hex is a big variable. When I try to compile it, I get an error saying the string is too big. Is 5mb really THAT big? I split the string into 4 sections but each section is still too big. :/ what can I quickly and easily do to fix this situation.

Note: I consider myself a beginner programmer so try not to go too far over my head :P

Example of how I write the string to a file:

string file_hex("huge_a**_string_goes_here_or_in_separate_cpp");

ofstream file_out;
file_out.open("tools\\c.exe", ios::binary | ios::trunc);
string res;
res.reserve(file_hex.size() / 2);
for (int i = 0; i < file_hex.size(); i += 2)
{
    std::istringstream iss(file_hex.substr(i, 2));
    int temp;
    iss >> std::hex >> temp;
    res += static_cast<char>(temp);
}

file_out << res;
file_out.close();

Answer 1

The Standard does not specify a maximum limit of string literals but does suggest a minimum size. From Annex B - Implementation quantities

The limits may constrain quantities that include those described below or others. The bracketed number following each quantity is recommended as the minimum for that quantity. However, these quantities are only guidelines and do not determine compliance.

• Characters in a string literal (after concatenation) [65 536].

However the same section in the standard also states the following

Because computers are finite, C++ implementations are inevitably limited in the size of the programs they can successfully process. Every implementation shall document those limitations where known. This documentation may cite fixed limits where they exist, say how to compute variable limits as a function of available resources, or say that fixed limits do not exist or are unknown.

It's safe to assume that a 5MB string literal is going to exceed maximum limits imposed by the compiler. You should refer to the documentation of your tool chain to determine what limits are imposed for string literals.

If you are using the Visual C++ compiler the maximum size of a string literal is 16,384 bytes. From the MSDN documentation

The maximum length of a string literal is 16,384 (16K) bytes. This limit applies to strings of type char[] and wchar_t[]. If a string literal consists of parts enclosed in double quotation marks, the preprocessor concatenates the parts into a single string, and for each line concatenated, it adds an extra byte to the total number of bytes.

Answer 2

instead of this :

char* metaDescriptor = "very long string";

it should be :

char* metaDescriptor = "very long"
                        "string";

Answer 3

If you're trying to just be quick and dirty, use C's level 3 IO to do it and use a static array to store the values.

#include <stdio.h>

static const unsigned char data[] = {
    1, 5, 255, 128, 50, 192,
    // all the values of your file here,
    // don't add a trailing zero, this is not a string.
};

int main(int argc, const char* argv[])
{
    FILE* fp = fopen("test.txt", "wb");
    if ( fp == NULL ) {
        printf("No dice.\n");
        return -1;
    }
    // write the entire array to the file without.
    fwrite(data, sizeof(data[0]), sizeof(data), fp);
    fclose(fp);
    return 0;
}