Face detection after background substraction using openCv

Question

I'm trying to improve face detection from a camera capture so I thought it would be better if before the face detection process I removed the background from the image, I'm using BackgroundSubtractorMOG and CascadeClassifier with lbpcascade_frontalface for face detection,

My question is: how can I grab the foreground image in order to use it as the input to face detection? this is what I have so far:

while (true) {
    capture.retrieve(image);

    mog.apply(image, fgMaskMOG, training?LEARNING_RATE:0);

    if (counter++ > LEARNING_LIMIT) {
        training = false;
    }

    // I think something should be done HERE to 'apply' the foreground mask 
    // to the original image before passing it to the classifier..

    MatOfRect faces = new MatOfRect();
    classifier.detectMultiScale(image, faces);

    // draw faces rect
    for (Rect rect : faces.toArray()) {
        Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 0, 0));
    }

    // show capture in JFrame
    frame.update(image);
    frameFg.update(fgMaskMOG);

    Thread.sleep(1000 / FPS);
}

Thanks

Answer 1

If you have the input image and the foreground mask, this is straight forward. In C++, I would simply add (just where you put your comment): image.copyTo(fgimage,fgMaskMOG);

I'm not familiar with the java interface, but this should be quite similar. Just don't forget to correctly initialize fgimage and reset it each frame.

Answer 2

I can answer in C++ using the BackgroundSubtractorMOG2:

You can either use erosion or pass a higher threshold value to the MOG background subtractor to remove the noise. In order to completely get rid of the noise and false positives, you can also blur the mask image and then apply a threshold:

// Blur the mask image
blur(fgMaskMOG2, fgMaskMOG2, Size(5,5), Point(-1,-1));

// Remove the shadow parts and the noise
threshold(fgMaskMOG2, fgMaskMOG2, 128, 255, 0);

Now you can easily find the rectangle bounding the foreground region and pass this area to the cascade classifier:

// Find the foreground bounding rectangle
Mat fgPoints;
findNonZero(fgMaskMOG2, fgPoints);
Rect fgBoundRect = boundingRect(fgPoints);

// Crop the foreground ROI
Mat fgROI = image(fgBoundRect);

// Detect the faces
vector<Rect> faces;
face_cascade.detectMultiScale(fgROI, faces, 1.3, 3, 0|CV_HAAR_SCALE_IMAGE, Size(32, 32));

// Display the face ROIs
for(size_t i = 0; i < faces.size(); ++i) 
{
    Point center(fgBoundRect.x + faces[i].x + faces[i].width*0.5, fgBoundRect.y + faces[i].y + faces[i].height*0.5);
    circle(image, center, faces[i].width*0.5, Scalar(255, 255, 0), 4, 8, 0);
} 

In this way, you will reduce the search area for the cascade classifier, which not only makes it faster but also reduces the false positive faces.