Eye detection within face

Question

I am trying to detect eyes which only fall within the face region and hence I did some minor alterations to the code:

 if( cascade )
    {
        double t = (double)cvGetTickCount();
        CvSeq* faces = cvHaarDetectObjects( small_img, cascade, storage,
                                            1.1, 2, 0|CV_HAAR_DO_CANNY_PRUNING,
                                            cvSize(30, 30) );
        t = (double)cvGetTickCount() - t;

        printf( "detection time = %gms\n", t/((double)cvGetTickFrequency()*1000.) );
        for( i = 0; i < (faces ? faces->total : 0); i++ )      
        {

            CvRect* r = (CvRect*)cvGetSeqElem( faces, i );

            CvMat small_img_roi;
            CvSeq* nested_objects;
            CvPoint center;
            CvPoint center_1;
            CvScalar color = colors[i%8];
            int radius,radius_1;
            center.x = cvRound((r->x + r->width*0.5)*scale);
            center.y = cvRound((r->y + r->height*0.5)*scale);
            radius = cvRound((r->width + r->height)*0.25*scale);
            cvCircle( img, center, radius, color, 3, 8, 0 );
            if( !nested_cascade )
                continue;
            else
                printf("not continuing!\n");
            cvGetSubRect( small_img, &small_img_roi, *r );
            nested_objects = cvHaarDetectObjects( &small_img_roi, nested_cascade, storage,
                                        1.1, 2, 0
                                        |CV_HAAR_DO_CANNY_PRUNING,                                        ,
                                        cvSize(0, 0) );
            for( j = 0; j < (nested_objects ? nested_objects->total : 0); j++ )
            {
                printf("start of nested objects loop!\n");
                CvRect* nr = (CvRect*)cvGetSeqElem( nested_objects, j );
                center_1.x = cvRound((r->x + nr->x + nr->width*0.5)*scale);
                center_1.y = cvRound((r->y + nr->y + nr->height*0.5)*scale);
                radius_1 = cvRound((nr->width + nr->height)*0.25*scale);
                if(center_1.x+radius_1<center.x+radius&& center.x-radius<center_1.x-radius_1 && center_1.y<center.y+radius&& center.y<center_1.y+radius )
                {
                cvCircle( img, center_1, radius_1, color, 3, 8, 0 );
                }
                else
                    printf("cant find thy eyes!\n");
            }

However, it wasn't very successful. In trying to debug it, I tried commenting off the part where they draw a circle for the face, which resulted in no circles at all being drawn. This led me to the conclusion that perhaps the part with nested objects was not working. Hence, I implemented several printfs in the code and monitored the console. But after observing the console, I came to the conclusion that indeed the part nested objects was not working. However, I am still clueless as to why this is so as the nested objects part was similar to the face detection portion. Hence, if the face detection portion works, shouldn't the nested object code work too?

(>_<)

Answer 1

Please provide more informations:

1. Which cascade file are you using?
2. What is the height and width of small_img?

About your code:

1. You are using nested_cascade variable without initializing it.

Generally - try to use haarcascade_mcs_eyepair_big.xml file from openCV with this parameters: (image, cascade, storage, 1.1, 3, 0, cvSize()) or even with smaller value of scaleFactor parameter.

I have done something similar during writing my BSc Thesis (Eyetracking system) and ended with quite easy solution. At the beginning i was trying to find inside face all 'objects' which can be eye and then decide which one is left eye and which one is right, but now i don't think it's a good solution.
Finally i decided to try to search for both eyes (pair - haarcascade_mcs_eyepair_big.xml file from opencv) at once inside face (haarcascade_frontalface_default.xml) and i find out that it's much better solution. It's faster, less complicated (you don't have to decide which object is the one you are looking for - it can be quite complicated if you try to have ragard previous positions of eyes and few other things) and easier to implement.
The accuracy was good enough for me and speed was quite high (whole system - about 10-25fps - it depends on few things). If you want to use your code for real-time eye detection i can give you few more details about optimization.

Answer 2

import  cv2

#loading cascade
eye_Cascade=cv2.CascadeClassifier('haarcascade_eye.xml')
face_Cascade=cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

def detect(gray,frame):
faces=face_Cascade.detectMultiScale(gray,1.3,5)
for (x,y,w,h) in faces:
    cv2.rectangle(frame,(x,y),(x+w ,y+h),(255,0,0),2)
    roi_gray=gray[y:y+h,x:x+w]
    roi_color=frame[y:y+h,x:x+h]
    eyes=eye_Cascade.detectMultiScale(roi_gray,1.1,3)
    for(ex,ey,ew,eh) in eyes:
        cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
return frame        

video_capture=cv2.VideoCapture(0)  #here 0 is used for inbuilt cam and 1 for 
external cam
while True :
   _,frame=video_capture.read()
   gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 
   canvas = detect(gray, frame)
   cv2.imshow('Face and Eye ', canvas)

   #exit when user pressed 'q'
   if cv2.waitKey(1) & 0xFF == ord('q'):   
    break

video_capture.release()
cv2.destroyAllWindows()