File sent using HttpPost corrupted and/or truncated

Question

I receive a file using the following code:

byte[] fileBytes;
....
JSONObject postJSON = new JSONObject();
postJSON.put("file_name", filename);
postJSON.put("client_id", clientID);
HttpPost post = new HttpPost(fileURL);
StringEntity se = new StringEntity( postJSON.toString(), "UTF-8");  
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
post.setEntity(se);
response = httpClient.execute(post);
fileBytes = EntityUtils.toByteArray(response.getEntity());

Using the debugger, I see that the response gets an entity 27136 bytes in length, which is the correct length of the test file, but the fileBytes array is only 11470 bytes long. Can anyone tell my why this truncation is taking place? When I try to get other files, a similar truncation takes place, so it is not a function of the specific file or a specific file length.

Using the following code, I get 11997 bytes for the same file:

StringBuilder stringBuilder = new StringBuilder("");
stringBuilder.append(EntityUtils.toString(response.getEntity()));
fileBytes = stringBuilder.toString().getBytes();

Reading from an InputStream, I get 12288 bytes:

fileBytes = new byte[1024];
InputStream inputStream = response.getEntity().getContent();
int bytesRead = 0;
while(true){
    bytesRead = inputStream.read(fileBytes);
    if (bytesRead <= 0)
        break;
....
}

Changing the encoding to UTF-16 gets me an internal server error.

I also tried the following:

InputStream inputStream = response.getEntity().getContent();
response.getEntity().getContentLength()];
while ((getByte = inputStream.read()) != -1) {
    bos.write(getByte);
}
bos.close();

This also gave me a file of 11470.

In all cases, the files are corrupted, and cannot be opened. When compared in a binary file viewer, the firs 11 bytes match, and then the files diverge. I could not find any pattern in the corrupted file.

Answer 1

OK, the answer is apparently that all of the above are fine. The problem was with the server, which was not configuring the data stream correctly: Content-type was text/plain for all files, rather than application/pdf, and so on as appropriate.

My first clue was when we put a text file on the server, and it came over successfully. At that point I started working with the server side, and we figured it out pretty quickly.

Bottom line, if you are working on a server/client application, the problem might not be on your side.

I should have mentioned various posts which helped my construct the various versions that I collected above:

including this

and this

My apologies to various other helpful people whose posts I also looked at and up-voted.