-Java- Data 'loss' in TCP (strange, since TCP doesn't allow the data to escape)

Question

first of all, I know there are already a couple of posts here about similar issues, but I've gone through them and their problems were not mine nor their solutions solve my problem. If anyone has a post which solves my question, please reply with the link and I'll go for it.

Now, the problem - I've got a client and a server. The client requests a file, the server sends it and then the client receives it, or at least that's how it should be. But it is not. What happens instead? The client receives the first 1024 bytes, then the next 1024 bytes, and then receives 436 bytes (I'm always using the same file so I always get the same results) and ends because it has received less than 1024 bytes and therefore it has deducted than it was the last read it had to perform, but it shouldn't be, because the server, when reading the file from its FileInputStream, reads much more bytes (much more pieces of 1024 bytes). I've tried everything I've imagined, but I've got no positive result: Always, the third piece, instead of being whole received, is just received up to the byte 436 (maybe what happens is that, from the third piece up to the penultimate are skipped for some reason, and then the last one is 436 bytes and is received, althought I don't think this is the case since the whole file isn't read in the server because the client closes the connection as soon as it thinks the transmission has finished, provoking an exception on the server that interrupts it from reading anything).

And here goes what I think is the key code:

Server

OutputStream put;
        try {
            ServerSocket serverSocket;
            try {
                serverSocket = new ServerSocket(DesktopClient.PORT_FOR_FILES);
            } catch (IOException ex) {
                JOptionPane.showMessageDialog(null, ex.toString(), "Error", JOptionPane.ERROR_MESSAGE);
                return false;
            }
            Socket socket;
            try {
                socket = serverSocket.accept();
                ActionWindow.println("Connection for files transference stablished - " + socket);
            } catch (Exception ex) {
                JOptionPane.showMessageDialog(null, ex.toString(), "Error", JOptionPane.ERROR_MESSAGE);
                return false;
            }
            put = socket.getOutputStream();
            BufferedReader requestedVideoBufferedReader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
            String requiredVideo = requestedVideoBufferedReader.readLine();
            while (!requiredVideo.matches("finish") && !requiredVideo.matches("errored")) {
                ActionWindow.println("The screen in the router " + ActionWindow.currentConn + " is asking for " + requiredVideo);
                if (new File(ActionWindow.pathToVideosFolder + requiredVideo + ActionWindow.videoFilesExtension).exists()) {
                    try (InputStream in = new FileInputStream(ActionWindow.pathToVideosFolder + requiredVideo + ActionWindow.videoFilesExtension)) {
                        byte[] buf = new byte[1024];
                        int len;
                        System.out.println("About to write");
                        while (true) {
                            len = in.read(buf);
                            System.out.println("len: " + len);
                            if (len > 0) {
                                put.write(buf, 0, len);
                            } else {
                                break;
                            }
                        }
                    }
                    ActionWindow.println("File " + requiredVideo + ActionWindow.videoFilesExtension + " sent.");
                } else {
                    ActionWindow.println("File " + requiredVideo + ActionWindow.videoFilesExtension + " couldn't be found in the local index.");
                    put.close();
                    socket.close();
                    serverSocket.close();
                    return false;
                }
                requiredVideo = requestedVideoBufferedReader.readLine();
            }
            if (requiredVideo.matches("errored")) {
                JOptionPane.showMessageDialog(null, "Se produjo un error en la transmisión de archivos. Los datos del servidor no han sido modificados.", "Error", JOptionPane.ERROR_MESSAGE);
            }
            put.close();
            socket.close();
            serverSocket.close();
            ActionWindow.println("Connecion finished.");
        } catch (IOException ex) {
            if (!(ex instanceof SocketException)) { //This means that the exception has not been caused by finishing the transference.
                JOptionPane.showMessageDialog(null, ex.toString(), "Error", JOptionPane.ERROR_MESSAGE);
                return false;
            } else {
                return true;
            }
        }
        return true;

Client

fileRequester.println(x);
            ServerDaemon.println("Requested " + x);
            File destFile = new File(ServerDaemon.pathToVideosFolder + x + ServerDaemon.videoFilesExtension);
            byte[] buf = new byte[1024];
            OutputStream streamForWritingToFile;
            try {
                streamForWritingToFile = new FileOutputStream(destFile);
            } catch (FileNotFoundException ex) {
                ServerDaemon.println(ex.toString());
                return false;
            }
            int len;
            try {
                while (true) {
                    len = streamFromServer.read(buf);

                    if (len > 0) {
                        streamForWritingToFile.write(buf, 0, len);
                    } else {
                        break;
                    }

                    if (len != 1024) {
                        break;
                    }
                }
            } catch (IOException ex) {
                ServerDaemon.println(ex.toString());
                if (ex instanceof SocketException) {
                    ServerDaemon.disconnect();
                }
                return false;
            }

            ServerDaemon.println("Received file " + x);

EDIT: As a temporal solution, I'm sending the file size through UDP so the receiver know how much data must it wait for. However, it's higly preferred to have it working through TCP, but it doesn't work (take a look at the comments in @PeterLawrey 's answer). This is what I've tried in order to make it work through TCP:

Receiver:

BufferedReader inFromClient;
int fileSize;
String receivedSizeMsg = null;
try {
      inFromClient = new BufferedReader(new InputStreamReader(socket.getInputStream()));
      receivedSizeMsg = inFromClient.readLine();
    } catch (IOException ex) {
      ServerDaemon.println(ex.toString());
    }
fileSize = Integer.parseInt(receivedSizeMsg.substring(4));

Sender:

DataOutputStream outToServer;
outToServer = new DataOutputStream(socket.getOutputStream());
outToServer.writeBytes("size" + new File(file+"\n");

SECOND EDIT: With TCP:

Receiver:

DataInputStream inFromClient;
int fileSize = 0;
inFromClient = new DataInputStream(socket.getInputStream());
fileSize = Integer.parseInt(inFromClient.readUTF().substring(4));

Sender:

DataOutputStream outToServer;
outToServer = new DataOutputStream(socket.getOutputStream());
outToServer.writeUTF("size" + new File(ActionWindow.pathToVideosFolder + requiredVideo + ActionWindow.videoFilesExtension).length());

Answer 1

You have a problem here.

                if (len != 1024) {
                    break;
                }

This assumes that data will arrive with the same length it is sent with which is not true. A blocking read() is only guaranteed to read one byte. Take this out of your code and it may work now.

For example, TCP often combines data into packets of around 1500 bytes. When you are reading the data, you can read it faster than it is sent so you might expect to read a block of 1024 bytes followed by 476 bytes (minus any TCP header)

---

EDIT: a simple send file and receive file method.

public static void sendFile(DataOutputStream out, String fileName) throws IOException {
    FileInputStream fis = new FileInputStream(fileName);
    try {
        byte[] bytes = new byte[8192];
        int len;
        out.writeInt((int) fis.getChannel().size());
        while ((len = fis.read(bytes)) > 0)
            out.write(bytes, 0, len);
    } finally {
        fis.close();
    }
}

public static void recvFile(DataInputStream in, String fileName) throws IOException {
    FileOutputStream fos = new FileOutputStream(fileName);
    try {
        byte[] bytes = new byte[8192];
        int left = in.readInt();
        int len;
        while(left > 0 && (len = in.read(bytes, 0, Math.min(left, bytes.length))) > 0) {
            fos.write(bytes, 0, len);
            left -= len;
        }
    } finally {
        fos.close();
    }
}

Answer 2

The others have given you the answer, basically, you aren't guaranteed to get data in a single packet. In networking, there are so many variables such as configuration (tcp_nodelay), packet size, etc. that it's better to do defensive programming.

Build a little protocol. Send say a 15 byte header that contains the size of the file and maybe a checksum. Any client who connects has to send that 15 byte header. Then your server can just sit in a loop and wait for data. You can even poll the socket (select) and time out if you don't get any more data for N number of seconds.

At the end, you can use the checksum to see if the file transmitted successfully.

Java World had an article about how to check the socket without blocking, but if your just doing a homework assignment or something small, it's probably not worth it.