Broken HTML - browsers don't downloads whole HTTP response from my webserver, CURL does

Question

Symptom

I think, I messed up something, because both Mozilla Firefox and Google Chrome produce the same error: they don't receive the whole response the webserver sends them. CURL never misses, the last line of the quick-scrolling response is always "</html>".

Reason

The reason is, that I send response in more part:

    sendHeaders();  // is calls sendResponse with a fix header
    sendResponse(html_opening_part);
    for ( ...scan some data... ) {
        sendResponse(the_data);
    } // for
    sendResponse(html_closing_part)

The browsers stop receiving data between sendResponse() calls. Also, the webserver does not close() the socket, just at the end.

(Why I'm doing this way: the program I write is designed for non-linux system, it will run on an embedded computer. It has not too much memory, which is mostly occupied by lwIP stack. So, avoid collecting the - relativelly - huge webpage, I send it in parts. Browsers like it, no broken HTML occurred as under Linux.)

Environment

The platform is GNU/Linux (Ubuntu 32-bit with 3.0 kernel). My small webserver sends the stuff back to the client standard way:

    int sendResponse(char* data,int length) {

        int x = send(fd,data,length,MSG_NOSIGNAL);
        if (x == -1) {
            perror("this message never printed, so there's no error \n");
            if (errno == EPIPE) return 0;
            if (errno == ECONNRESET) return 0;

            ... panic() ... (never happened) ...

        } // if send()

    } // sendResponse()

And here's the fixed header I am using:

    sendResponse(
        "HTTP/1.0 200 OK\n"
        "Server: MyTinyWebServer\n"
        "Content-Type: text/html; charset=UTF-8\n"
        "Cache-Control: no-store, no-cache\n"
        "Pragma: no-cache\n"
        "Connection: close\n"
        "\n"
    );

Question

Is this normal? Do I have to send the whole response with a single send()? (Which I'm working on now, until a quick solution arrives.)

Answer 1

Gotcha!

As @Jim adviced, I've tried sending same headers with CURL, as Mozilla does: fail, broken pipe, etc. I've deleted half of headers: okay. I've added back one by one: fail. Deleted another half of headers: okay... So, there is error, only if header is too long. Bingo.

As I've said, there're very small amount of memory in the embedded device. So, I don't read the whole request header, only 256 bytes of them. I need only the GET params and "Host" header (even I don't need it really, just to perform redirects with the same "Host" instead of IP address).

So, if I don't recv() the whole request header, I can not send() back the whole response.

Thanks for your advices, dudes!

Answer 2

If you read RFC 2616, you'll see that you should be using CR+LF for the ends of lines.

Aside from that, open the browser developer tools to see the exact requests they are making. Use a tool like Netcat to duplicate the requests, then eliminate each header in turn until it starts working.