Sending a Mat object over socket from c++ to Java

Question

I'm interested on sending a video stream from a C++ server to a Java client. The C++ server is written with boost/asio.
I was thinking of using opencv in order to easily connect to a computer's usb camera. However, I'm having troubles with sending Mat object over socket.

The only way ,that I encountered, to do so is this way:

Mat frame;
...
int  imgSize = frame.total()*frame.elemSize();
bytes = send(clientSock, frame.data, imgSize, 0))

but I cannot send it with boost this way, since it has to be in one of these forms: http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference/basic_raw_socket/send_to.html

So I thought of converting frame.data ,imgSize to string, and sending a container of these two buffers, but I don't think this is the right way. And even if it is, how can I "rebuild" the image in the Java client?

Any idea would be appreciated!

Answer 1

On the server side, encoding to jpeg:

vector<uchar> buff;
vector<int> params;

params.push_back(cv::IMWRITE_JPEG_QUALITY);
//80 keeps the image size under 64KB ,ON MY CAMERA, so I can send the image in one piece
params.push_back(80);
cv::imencode(".jpg", frame, buff, params);

boost::asio::ip::udp::socket* _socket;
...
_socket->send_to(boost::asio::buffer(buff), some boost::asio::ip::udp::endpoint, 0, some boost::system::error_code);

On the client side:

DatagramPacket receivePacket; //stores the data sent from server
...
byte[] receivedImage = receivePacket.getData();
MatOfByte mob = new MatOfByte(receivedImage);
Mat img = Highgui.imdecode(mob, Highgui.IMREAD_UNCHANGED);

Answer 2

You could try to encode your image in Base64 to send them over sockets using a protocol like XML or JSON.

Base64 is a group of similar binary-to-text encoding schemes that represent binary data in an ASCII string format

Probably this is not the most efficient method but it could be the easiest for you at this moment. You can find more about Base64 on the wiki page and you can find more info about converting an OpenCV Mat to Base64 here. There are also a few tutorials on how to display Base64 images in Java.