Encode image in JPG with OpenCV avoiding the ghost effect

Question

I have an application (openCV - C++) that grab an image from webcam, encode it in JPG and trasmitt it from a Server to Client. Thwebcam is stereo so actually I have two image, LEFT and RIGHT. In the client, when I recieve the image I decode it and I generate an Anaglyph 3D Effect. For do this I use the OpenCV... Well I encode the image in this way:

 params.push_back(CV_IMWRITE_JPEG_QUALITY);
 params.push_back(60); //image quality
 imshow(image); // here the anagliphic image is good!
 cv::imencode(".jpg", image, buffer, params);

and decode in this way:

 cv::Mat imageRecieved = cv::imdecode( cv::Mat(v), CV_LOAD_IMAGE_COLOR );

What I see is that this kind of encode generate in the Anaglyph image a "ghost effect" (artifact?) so there is a bad effect with the edges of the object. If look a door for example there a ghost effect with the edge of the door. I'm sure that this depends of the encode because if I show the Anaglyph image before encode instruction this work well. I cannot use the PNG because it generate to large image and this is a problem for the connection between the Server and the Client.

I look for the GIF but, if I understood good, is nt supported by the cv::encode function.

So there is another way to encode a cv:Mat obj in JPG withou this bad effect and without increase to much the size of the image?

Answer 1

If your server is only used as an image storage, you can send to the server the 2 original stereo images (compressed) and just generate the Anaglyph when you need it. I figure that if you fetch the image pair (JPEG) from the server and then generate the Anaglyph (client-side), it will have no ghosting. It might be that the compressed pair of images combined is smaller than the Anaglyph .png.

Answer 2

I assume the anaglyph encoding is using line interlacing to combine both sides into one image.

You are using JPEG to compress the image. This algorithm optimized to compress "photo-like" real world images from cameras, and works very well on these.

The difference of "photo-like" and other images, regarding image compression, is about the frequencies occurring in the image.
Roughly speaking, in "photo-like" images, the high frequency part is relatively small, and mostly not important for the image content.
So the high frequencies can be safely compressed.

If two frames are interlaced line by line, this creates an image with very strong high frequency part.

The JPEG algorithm discards much of that information as unimportant, but because it is actually important, that causes relatively strong artefacts.

JPEG basically just "does not work" on this kind of images.

If you can change the encoding of the anaqlyph images to side by side, or alternating full images from left and right, JPEG compression should just work fine.

Is this an option for you?

If not, it will get much more complicated. One problem - if you need good compression - is that the algorithms that are great for compressing images with very high frequencies are really bad at compressing "photo-like" data, which is still the larger part of your image.

Therefore, please try really hard to change the encoding to be not line-interlacing, that should be about an order of magnitude easier than other options.