Efficient way to send an image over socket in Java

Question

I'm a bit of a Java noob, and I have read some basics about sockets and I can successfully send images over socket using ImageIO, but I want to reduce the amount of data that is sent. Ultimately I want the image (screen capture) to be send as fast as possible with the smallest possible file size.

Right now, I have imageIO set up as such;

DataInputStream in=new DataInputStream(client.getInputStream());

DataOutputStream out = new DataOutputStream(client.getOutputStream());

ImageIO.write(captureImg(),"JPG",client.getOutputStream());

And the receiver:

BufferedImage img=ImageIO.read(ImageIO.createImageInputStream(server.getInputStream()));

File outputfile = new File("Screen"+(date.toString())+".jpg");

ImageIO.write(img, "jpg", outputfile);

In case you're wondering, this is my method that is used to take the image.

Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());              
BufferedImage capture = new Robot().createScreenCapture(screenRect);

I have heard about Byte arrays, where you can send the bytes then draw the image at the other end. However I'm not sure if this is more efficient.

Any help would be greatly appreciated, please ask if you would like me to add any extra info or code for the byte array!

Thanks.

EDIT: Patrick:

ByteArrayOutputStream bScrn = new ByteArrayOutputStream(); 
ImageIO.write(captureImg(), "JPG", bScrn); 
byte imgBytes[] = bScrn.toByteArray();


out.write((Integer.toString(imgBytes.length)).getBytes());
out.write(imgBytes,0,imgBytes.length);

Answer 1

There already has been an extensive discussion in the comments, but to summarize a few points that I find important:

You have a trade-off between several criteria:

• Minimize network traffic
• Minimize CPU load
• Maximize image quality

You can reduce the network traffic with a high image compression. But this will increase the CPU load and might reduce the image quality.

Whether it reduces the image quality depends on the compression type: For JPG, you can make the image arbitrarily small, but the quality of the image will then be ... well, arbitrarily bad. For PNG, the image quality will stay the same (since it is a lossless compression), but the CPU load and the resulting image size may be greater.

The option of ZIPping the image data was also mentioned. It is true that ZIPping the JPG or PNG data of an image will hardly reduce the amount of data (because the data already is compressed). But compressing the raw image data can be a feasible option, as an alternative to JPG or PNG.

Which compression technique (JPG, PNG or ZIP) is appropriate also depends on the image content: JPG is more suitable for "natural" images, like photos or rendered images. These can withstand a high compression without causing artefacts. For artifical images (like line drawings), it will quickly cause undesirable artefacts, particularly at sharp edges or when the image contains texts. In contrast to that: When the image contains large areas with a single color, then a compression like PNG (or ZIP) can reduce the image size due to the "run length compression" nature of these compression methods.

I already made some experiments for such an image transfer quite a while ago, and implemented it in a way that easily allowed tweaking and tuning these parameters and switching between the different methods, and comparing the speed for different application cases. But from the tip of my head, I can not give a profound summary of the results.

BTW: Depending on what you actually want to transfer, you could consider obtaining the image data with a different technique than Robot#createScreenCapture(Rectangle). This method is well-known to be distressingly slow. For example, when you want to transfer a Swing application, you could let your application directly paint into an image. Roughly with a pattern like

BufferedImage image = new BufferedImage(w,h,type);
Graphics g = image.getGraphics();
myMainFrame.paint(g);
g.dispose();

(This is only a sketch, to show the basic idea!)

Additionally, you could consider further options for increasing the "percieved speed" of such an image transfer. For example, you could divide your image into tiles, and transfer these tiles one after another. The receiver will possibly appreciate it if the image would at least be partially visible as quickly as possible. This idea could be extended further. For example, by detecting which tiles have really changed between two frames, and only transfer these changed tiles. (This approach could be extended and implemented in a rather sophisticated way, by detecting the "minimum regions" that have to be transferred)

However, for the case that you first want to play around with the most obvious tuning parameter: Here is a method that allows writing a JPG image with a quality value between 0.0 and 1.0 into an output stream:

public static void writeJPG(
    BufferedImage bufferedImage,
    OutputStream outputStream,
    float quality) throws IOException
{
    Iterator<ImageWriter> iterator =
        ImageIO.getImageWritersByFormatName("jpg");
    ImageWriter imageWriter = iterator.next();
    ImageWriteParam imageWriteParam = imageWriter.getDefaultWriteParam();
    imageWriteParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
    imageWriteParam.setCompressionQuality(quality);
    ImageOutputStream imageOutputStream =
        new MemoryCacheImageOutputStream(outputStream);
    imageWriter.setOutput(imageOutputStream);
    IIOImage iioimage = new IIOImage(bufferedImage, null, null);
    imageWriter.write(null, iioimage, imageWriteParam);
    imageOutputStream.flush();
}