Why is the last chunk of HTTP download really slow?

Question

i have content length available in the beginning of the download. so i know how much bytes i need to request for. i download in chunks of 1024 bytes. in the last chunk i request for the number of bytes remaining. i am using the read function. but the last chunk takes a lot of time to arrive. is it normal?

Answer 1

Thanks i figured out the problem. my code uses an application level buffer and hence the last chunk takes a long time to download. i have been using robust io functions given in Bryant's book. i had studied that code some time back and had forgotten about it. i revised the code and found buffering being used by the code.

bryant's book - rio functions

Another mistake that i was making was using HTTP/1.1. HTTP 1.0 causes the server to close the connection once it has transmitted the data. So that solved the problem.

Answer 2

Maybe your last chunk is not big enough to flush the buffer.

You could check how to flush the fd and do it manually after last chunk is sent.

Answer 3

No. My guess is that the server is missing a call to flush() so the output hangs in some buffer until it gets a timeout (and then, the server will flush).