Advice and Help Troubleshooting Client/Server Interaction for Cellular Modem Project

Question

I need some advice and help troubleshooting a client/server interaction for a cellular modem project I'm working on. The client and server are written in Java.

The questions I need some advice on are as follows:

(1) I'm seeking advice on whether the approach I'm taking is going to scale up for large files (Code to follow) especially in the context of a cellular network where the network can drop out unexpectedly. Large files being ~1GB. The large file represent an image that is transferred from the server to client. This represents the worst case scenario. The base case contains files that are relatively small, containing GPS Data and Time Stamp info. These files are likely in the KB to a couple of MB range and transferred often.

(2) Need advice on troubleshooting the client/server code. Even if this approach isn't going to work for larger files, I would like to get the code working for the base case to support a proof of concept test in the near future. Software updates are not required for the proof of concept.

Background on the client/server interaction. The client contacting the server. The server detects the client request which starts a new thread to handle the client. The client transmits a serilaize Data Packet. The Data Packet contains some header info (file size, crc, file type) and a data payload. Upon receiving the data packet object, the server verifies the crc and file size match the values include in the header. The server responds with a data packet object which indicates whether the transmission was valid or not. If the client receives a valid response from the server the client sends a good bye packet to close the session. If the response was invalid from the server, the client will resend the data and will eventually quit after x failed attempts. Eventually, the client will get a set of instructions from the server which can instruct it to upload a log file, download a new .jre file, or even a new firmware image.

Here is the error that I'm getting on the server code:

Feb 16, 2013 7:36:40 AM noaa.logbook.server.ServerConnectionHandler run
SEVERE: null
java.io.EOFException
at
java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at noaa.logbook.server.ServerConnectionHandler.run(ServerConnectionHandler.java:69)
    at java.lang.Thread.run(Thread.java:662)

The data arrives as the server and it looks exception occurs on the next time I tried to read an object on the objectinputstream, which is the goodbye packet.

Client Code:

public static boolean clientTransmit(Socket sockToServer, String fileName, int dataPacketType, String id, String fileTimeStamp)    {
ObjectOutputStream oos = null;
ObjectInputStream ois = null;
//TCPConnection tcpConn = null;
DataPacket packet = null;

File inputFile = new File(fileName);
boolean fileExists = inputFile.exists();

int size = 0;
int failedAttempts = 0;
String contents = null;
byte[] ref = null;
boolean success = false;
boolean bye = false;
try
{
  sockToServer.setSoTimeout(5000);

  if ((sockToServer.isConnected()) && (fileExists)) {
    System.out.println("LogBookClientCommunications: Connected to Server");
    System.out.print("Stage 0");
    contents = readFile(fileName);

    packet = LogBookUtilities.packageDataPacket(DataPacket.UPLOAD_DATA, contents,    LogBookClient.SOFTWARE_VERSION, LogBookClient.serialNumber);

    oos = new ObjectOutputStream(sockToServer.getOutputStream());
    oos.writeObject(packet);
    oos.flush();
    System.out.println("LogBookClientCommunications: Sending DataPacket");

    ois = new ObjectInputStream(sockToServer.getInputStream());

    while(!success && failedAttempts < 3) {
      Object object = ois.readObject();
      if (object instanceof DataPacket) {
        System.out.println("LogBookClientCommunications: Received a DataPacket Object");
        DataPacket inPacket = (DataPacket)object;
        byte[] compressedByteRef = inPacket.getDataArray();
        boolean sizeValid = verifySize(compressedByteRef, inPacket.getLength());
        boolean crcValid = verifyCRC(inPacket);
        if ((sizeValid) && (crcValid)) {
            System.out.println("LogBookClientCommunications: Size & CRC Valid");
            String uncompressed = new String(uncompress(compressedByteRef));
            String[] strRef = lookupResponsePairs(dataPacketType);

            if (uncompressed.equals(strRef[0])) {
                success = true;
                System.out.println("LogBookClientCommunications: File arrived uncorrupted");
                //tcpConn.disconnect();
            } else if (uncompressed.equals(strRef[1])) {
                success = false;
                failedAttempts++;
                System.out.println("LogBookClientCommunications: File arrived corrupted");
            }
        } else {
            success = false;
            failedAttempts++;
            if (sizeValid)
                System.out.println("LogBookClientCommunications: CRC InValid");
            else
                System.out.println("LogBookClientCommunications: Size InValid");
        }
      }//end if object instanceof
      else {
        System.out.println("LogBookClientCommunications: Not a DataPacket Object");
        failedAttempts++;
      }
    }//while
    //Close Connection by sending bye
    System.out.println("LogBookClientCommunications: Sending Good Bye...");
    DataPacket goodbye  = LogBookUtilities.packageDataPacket(DataPacket.RESPONSE, quit", LogBookClient.SOFTWARE_VERSION, LogBookClient.serialNumber);
    oos.writeObject(goodbye);
    oos.flush();
  }
  else
  {
    System.out.println("LogBookClientCommunications: Failed to Connect or File Did Not Exist");
    success = false;
  }
}
catch (ClassNotFoundException ex) {
  Logger.getLogger(LogBookUtilities.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
  Logger.getLogger(LogBookUtilities.class.getName()).log(Level.SEVERE, null, ex);
} finally {
  try {
    oos.close();
    ois.close();
    sockToServer.close();
  } catch (IOException ex) {
    Logger.getLogger(LogBookUtilities.class.getName()).log(Level.SEVERE, null, ex);
  }
}
return success;
}

Server connection handler code:

public void run()
{
ObjectInputStream ois = null;
ObjectOutputStream oos = null;
int failedAttempts = 0;
boolean success = false;
boolean sendResponse = false;
Socket soc = getSocket();
Object obj = new Object();
long time = System.currentTimeMillis();

DataPacket inPacket = null;
DataPacket outPacket = null;
try {
  System.out.println("Server Connection Handler: Receiving Connection From - " + soc.getRemoteSocketAddress());
  soc.setSoTimeout(15000);
  oos = new ObjectOutputStream(soc.getOutputStream());
  oos.flush();

  ois = new ObjectInputStream(soc.getInputStream());

  if (ois == null | oos == null) {
      System.out.println("Server Connection Handler: Successfull Opened Streams");
      if (ois == null) { System.out.println("Server Connection Handler: ObjectInputStream Failed to Open");}
      else {System.out.println("Server Connection Handler: ObjectOutputStream Failed to Open"); }
  }
  while (true) {
      inPacket = (DataPacket)ois.readObject();
      boolean validPacket = LogBookUtilities.isPacketValid(inPacket);
      if (validPacket) {
          if(inPacket.getField() == DataPacket.RESPONSE) {
              byte[] ref = inPacket.getDataArray();
              String data = LogBookUtilities.uncompress(ref);
              if (data.equalsIgnoreCase("bye")) {
                  System.out.println("Server Connection Handler: Bye....");
                  break;
              }
          }
          else if (inPacket.getField() == DataPacket.UPLOAD_DATA) {
              System.out.println("Server Connection Handler: Writing data to file");
              LogBookUtilities.processClientPacket(inPacket);
              System.out.println("Server Connection Handler: File Successfully Transfered");
              outPacket = LogBookUtilities.makeResponse(inPacket.getField(), true, LogBookServer.SOFTWARE_VERSION, LogBookServer.ID);
              sendResponse = true;
          }
      } 
      else {
          if (inPacket.getField() == DataPacket.UPLOAD_DATA) {
                sendResponse = true;
                outPacket = LogBookUtilities.makeResponse(inPacket.getField(), true, LogBookServer.SOFTWARE_VERSION, LogBookServer.ID);
          }
      }

      if (sendResponse) {
        oos.writeObject(outPacket);
        oos.flush();
      }

  }//end while


    }
    catch (ClassNotFoundException ex) {
      Logger.getLogger(ServerConnectionHandler.class.getName()).log(Level.SEVERE, null,     ex);
    }
    catch (IOException ex) {
      Logger.getLogger(ServerConnectionHandler.class.getName()).log(Level.SEVERE, null, ex);
    }
    finally {
      try {
        ois.close();
        oos.close();
        soc.close();
      }
      catch (IOException ex) {
        Logger.getLogger(ServerConnectionHandler.class.getName()).log(Level.SEVERE, null, ex);
      }
    }
    }

Server code:

public class LogBookServer
{
public static final String ID = "666666";
public static final String SOFTWARE_VERSION = "0.02";
public static final String VFILE = "vFile";
public static final String DASH = "-";
public static final String DATEXT = ".dat";
public static final int FAILED_THRESHOLD = 3;
private int port = 6767;
private String ip = "";

public int getListeningPort() {
  return this.port;
}

public void setListeningPort(int port) {
  this.port = port;
}

public void run()
  throws Exception
{
Selector acceptSelector = SelectorProvider.provider().openSelector();

ServerSocketChannel ssc = ServerSocketChannel.open();
ssc.configureBlocking(false);

InetAddress lh = InetAddress.getLocalHost();
InetSocketAddress isa = new InetSocketAddress(lh, this.port);
ssc.socket().bind(isa);

SelectionKey acceptKey = ssc.register(acceptSelector, 16);

int keysAdded = 0;

while ((keysAdded = acceptSelector.select()) > 0)
{
  Set readyKeys = acceptSelector.selectedKeys();
  Iterator i = readyKeys.iterator();

  while (i.hasNext()) {
    SelectionKey sk = (SelectionKey)i.next();
    i.remove();

    ServerSocketChannel nextReady = (ServerSocketChannel)sk.channel();

    Socket s = nextReady.accept().socket();

    handleConnection(s);
  }
 }
}

void handleConnection(Socket socket)
{
  System.out.println("hadling connection....");
  ServerConnectionHandler connectionHandler = new ServerConnectionHandler(socket);
  new Thread(connectionHandler).start();
}
}

Answer 1

In your client you have (split to diff lines for readability):

DataPacket goodbye  = 
    LogBookUtilities.packageDataPacket(DataPacket.RESPONSE, 
                                       "quit", 
                                       LogBookClient.SOFTWARE_VERSION,
                                       LogBookClient.serialNumber);

Then in your server you have:

if (data.equalsIgnoreCase("bye")) {

Which one of these is not like the other? ;)

Your server reads the "goodbye" packet, but doesn't recognize it, then loops again and tries to read from a closed socket. Presto, IOException.

As for your question of "scalability" ... it's not so much that as efficiency. If you're worried about the network dropping out from under you, sending serialized objects probably isn't the way to go; there's no way to resume - a partial send has to be completely resent and if it's a gig of data as you state ... that's bad. You are better off using the write() methods of OutputStream with a reasonable buffer size. This will allow you to keep track of the data that's been sent and resume a transfer once the network comes back (this will require some logic being implemented between your client and server, obviously, so that you can figure out what the server has already received in the case of a network failure).