openCV - videocapture of webcam - latency issue

Question

So I'm actually making a project of augmented reality. I use openCV to take picture from 2 camera.

Those camera arent really efficient, I think their max fps is around 30 fps.

If I refresh the camera frame (by the read method) in the programm at each frame the fps of the application is about 25 fps. If I don't refresh it It's about 55 fps.

I suppose that this latency is because openCV wait for a new frame to be generated by the cameras before going to the next step of the program.

But I need at least all virtual object to be rendered at 55 fps for imersion. Is their a way to say openCV to jump to the next call if there's no frame in the videoCapture object?

And if there's no way is there an other cross platform API for camera control more efficient?

Thx!

Answer 1

I never use OpenCV in C++, but I think this is the same. I am using OpenCV4Android and I need to do something when a frame comes in, it will acturally slow the fps if you put your procedure in the onCameraFrame() function(I guess its like the read() function in C++) because only when the frame return, the next frame is coming in.

My solution is use another thread to process the frame. what you can do in your read() function is setting the flag to indicate the frame is in videoCapture object or not, then use the process thread to check the flag, if there is, process it. The fps will be better.