OpenCV/FFMpeg image capture problems

Question

I'm trying to capture images from an IP camera in real time. The stream works perfectly well in VLC, but OpenCV's cvQueryFrame() seems to jumble and corrupt the incoming images to the point of no recognition.

Again, capturing from file works fine, but not a live stream. In case it makes a difference, I'm using an rtsp connection URL; I've also tried this with two different camera models (different brands), and the problem remains.

Besides, the (I'm assuming) codec is outputting several errors of the following kind: Error at MB: 1746 and concealing 6000 DC, 6000 AC, 6000 MV errors.

What can I do?

Update: The first error in the sequence is always cannot parallelize deblocking type 1, decoding such frames in sequential order

Update 2: Alright, it seems that OpenCV/FFMPEG has an issue with rtsp/h264 streams. I've tried the Qt Phonon library, which also doesn't work, and I've given the Live555 library a quick overview. This last appears to work, in the sense that everyone says it does, and the application example (OpenRTSP) in fact plays my stream well. However, to be quite honest, getting to grips with the Live555 code seems like a lengthy affair which I can hardly afford right now. Barring any other alternative, I guess I'll have to go that route.

Is there any other solution that comes to mind?

Update 3: I got the test RTSP client from the Live555 code to work, so I know how to extract h264 frame information from a stream, but now I need to recombine that frame information into actual displayable frames, which doesn't seem like something straightforward! Anyone familiar with Live555 know how to do this? Thanks.

Answer 1

It seems you need an extra software layer to capture the stream packets and reconstruct the frames locally, and then feed them to openCV. You can easily achieve this with libVLC. This would also avoid codec problems since you can parse almost all codecs with libVLC and then feed raw frames to openCV.

Answer 2

I don't know if it helps (since I'm not an experienced c++ dev), but I've recently managed to get a stream from an IP Camera. Here's a quick test:

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <stdio.h>

using namespace cv;
using namespace std;

int main(int, char**)
{
    VideoCapture ipCam;
    Mat frame;
    const string LOCATION = "rtsp://192.168.0.200:554/rtsph264vga";

    if(!ipCam.open(LOCATION)) {
        cout << "Error opening video stream or file" << endl;
        return -1;
    }

    for(;;) {
        if(!ipCam.read(frame)) {
            cout << "No frame" << endl;
            waitKey();
        }
        imshow("cam", frame);
        if(waitKey(1) >= 0) break;
    }

    return 0;
}

Before heading to c++ I've setup the camera to export to H264 VGA (as it wasn't enabled by default on the cam I'm working with) and made sure I've got the stream running in VLC. I'm using OpenCV 2.4.1 with fffmpeg enabled. As far I understand the ffmpeg integration with OpenCV is available from OpenCV 2.0 upwards.

I did run into a few issues when I had to integrate merge the cv code with other c++ code as I have OpenCV and ffmpeg + dependencies built for 64bit arch. and the other code was relying on many 32bit libraries. The VideoCapture class is part of the highgui lib and that mainly the one you need to worry about. If it's not compiled with ffmpeg support you will get an error or a warning as VideoCapture won't be able to transcode the content.

Not sure it it's the best option, but you could try to stream/transcode the stream from VLC (by ticking Streaming/Saving in the Open Source/Network tab)

Answer 3

Here is a code snippet, that I used to capture frames from WebCam. It worked for me, Hope it works for you as well...

int main(int argc, char* argv[])
{
    CvCapture *capture = NULL;
    IplImage* frame=NULL;
    int key =0;
    capture = cvCaptureFromCAM(0);

    if (!capture)   {
        printf("Cannot initailize webcam");
        return 1;
    }

    cvNamedWindow("result",CV_WINDOW_AUTOSIZE);

    while(key != 'q')
    {
        frame=cvQueryFrame(capture);

        if(!frame) break;

        cvShowImage("result",frame);
        key=cvWaitKey(10);
        frame=NULL;
    }
    cvDestroyWindow("result");
    cvReleaseCapture(&capture);
    return 0;
}

Answer 4

For OpenCV 2.3.1 I have written this code and it works normally, that is, I get the images from the camera feed.

VideoCapture cap(0);
if(!cap.isOpened())
{
    cout<<"Camera is not connected"<<endl;
    getchar();
} 
namedWindow("Camera Feed",1);
for(;;)
{
    Mat frame;
    cap >> frame;
    imshow("Camera Feed", frame);
    if(!frame.empty())
        detectAndDisplay(frame);
    else
        cout<<"No frame as input"<<endl;
    int c=waitKey(10);
    if(c==27)
        break;
}
return 0;

As you can see, it takes the input and displays it continuously and exits if you press ESC on your keyboard. Here's the documentation for CV 2.1 which has the same set of commands as CV 2.3. The commands changed from 2.4 I guess, although I am not too sure about it. Hope it helps.:)