It sounds as if you're not starting to write to disk until you've read the last byte from HTTP. That has two undesirable effects:
- You're putting a load of data into your server's memory footprint
- You're pushing all the disk IO to the end of the request handling, causing a pause that looks bad from the client perspective. In extreme cases, the client may time-out.
Instead, you need to be streaming. As soon as you have some data from the client, start writing to disk. When the client sends its final chunk, you only need to handle that small chunk before sending the 201 response.
If the API you're using doesn't allow you to consume HTTP entities in a streaming manner, I'm afraid you need to look for a different API.
For example, in the Java Servlet API, you can get an InputStream for the request content:
InputStream content = request.getInputStream();
// do something better than this
OutputStream out = new FileOutputStream("content.txt");
byte[] buffer = new byte[1024];
int len;
while ((len = in.read(buffer)) != -1) {
// whatever processing you want here
out.write(buffer, 0, len);
}
// now you can send the 201
It's important to understand that when you get the InputStream
in this example, your server hasn't started reading the content off the TCP stack yet (*). Only the headers have been read. If it's a large file, it's very likely that the client is still sending at this moment. You read a chunk, write a chunk, in a loop.
(*) That's a simplification. It may be in a buffer. But conceptually, you can ignore that