Find the differences between two images for screen sharing apps

Question

Hi I'm writing a client/server remote viewer (desktop sharing) application where screenshots of the desktop are sent across the network over a socket. I'd like to reduce the size of the transfer by getting the difference between the two images and then sending the difference. on the other hand difference will be merge with previous image at other end.

so anyone please guide me how could i accomplish this job. still now i send every time a complete image of the screen over the network programatically and program at other end just show that image. i feel huge data is getting pass over the network and screen update rate at the other end is slow. so please show me good way how to compare between two images and send only difference to other end. also tell me how to merge the difference with actual image at other end.

1) lots of free code and library is available for image comparison but i just do not understand which one i should use and which one would very faster for comparison. so just guide me regarding this.

2) and the most important part is how to send difference only over the network and merge the difference with actual image at other end

i tried lot to get some info regarding my point 2 but got nothing similar. no article i found who can guide me that how to send difference only over the network and merge the difference with actual image at other end

so i am looking for in-depth discussion for my point 2 specially. thanks

Answer 1

You will have to follow three steps:

1. Create a difference (DIFF) of the two consecutive images. Comparing the two images pixel per pixel will be very time consuming. You should utilize a well-established library like OpenCV; check out the Emgu CV library (http://www.emgu.com/) for .NET: AbsDiff() should be the method you're looking for. The result will be something like DIFF = IMG2 - IMG1.
2. Send the DIFF through the network. It's still a full image, but JPEG or PNG will utilize its full compression capability assuming it is a mainly black image, i.e. few changes. So these are actually three substeps: Compress - Send - Decompress.
3. Apply the DIFF on the present image. The recipient can calculate the next image IMG2 = DIFF + IMG1. This can be performed using EmguCV's Add method.

Answer 2

I know am very late responding but I found this question today

I have done some analysis on Image Differencing but the code was written for java. Kindly look into the below link that may come to help

How to find rectangle of difference between two images

The code finds differences and keeps the rectangles in a Linkedlist. You can use the linkedlist that contains the Rectangles to patch the differences on to the Base Image.

Cheers !

Answer 3

What about my approach, I am not sure it will be useful, I would like someone give some hint I am approaching in the right direction with this.

I consider streaming of desktop to mobile phone. I would use server-client model with network TCP sockets. I consider such pattern (protocol).

SERVER:

0)

SCREEN IMAGE - represented as unsigned char *rgba_array IMAGE_SIZE -> width, height

1)

Send image size width, height to Client.

2)

init rgba_array with length <= width x height x 4 bytes (32bits images) and all array zeroed.

0000 0000 0000 0000 0000 0000 0000 0000 (example pixel - 32 bits)

3)

if I have new image of screen (screenshot? or other way), I am converting it to unsigned char *new_rgba_array and than make a XOR operation:

rgba_array XOR new_rgba_array => diff_rgba_array 

4)

than I am only sending to the client this diff_rgba_array COMPRESSED? JPEG?

5) go back to point 3) on server side.

CLIENT:

0)

get width and height -> init rgba_array with length <= width * height * 4 and zeroed it

1)

get diff_rgba_array that represents changes in comparison with previous image and DECOMPRESS it? JPEG?

0011 1010 0110 0111 1110 0000 0100 0000 (example pixel difference)

0 - bit means NO CHANGE

1 - bit means CHANGE to opposed value

2)

apply changes received from server in diff_rgba_array to rgba_array on client side. I think I should make next XORing. Like this

  10011101   10010101

  00001111   11111111 (XOR) 

= 10010010   01101010 

3)

go back to point 1) on client side