how do i know which eye detection result is the best

Question

When detecting eyes using HaarDetectObject() function, we get the results (detectedObjects) like this:

[((110, 47, 44, 44), 11), ((27, 45, 54, 54), 31)] 
[((114, 49, 40, 40), 10), ((28, 50, 47, 47), 17)] 
[((31, 50, 47, 47), 18), ((106, 47, 44, 44), 3)] 
[((106, 46, 44, 44), 38), ((29, 53, 45, 45), 35), ((18, 30, 74, 74), 4)] 
[((98, 54, 36, 36), 7), ((108, 48, 45, 45), 31), ((26, 52, 49, 49), 49)] 
[((109, 46, 45, 45), 28), ((29, 50, 49, 49), 38)] 
[((109, 46, 44, 44), 29), ((27, 47, 53, 53), 27), ((123, 45, 48, 48), 3)] 
[((108, 46, 44, 44), 35), ((28, 50, 49, 49), 33)] 
[((109, 47, 45, 45), 34), ((28, 53, 47, 47), 35), ((126, 50, 44, 44), 3)] 
[((109, 47, 44, 44), 29), ((29, 47, 51, 51), 39)] 
[((106, 47, 42, 42), 49), ((27, 49, 49, 49), 30)] 
[((31, 54, 42, 42), 13), ((110, 46, 44, 44), 4), ((125, 46, 44, 44), 4), ((27, 35, 66, 66), 6)] 

and so on

what i do know is :

(110, 47, 44, 44), 11) ,which is x, y, w, h, ?

x,y are the co-ordinates of the top-left corner of detected 'eye' in 'eyes' and w,h are width and height to determine the detected eye region, so that's in cvRect format

So as you see, the last detection has 4 eyes - thus i ask, how do i know which two of them are my perfectly detected eyes? I mean, seeing the eyes[i][0][0] for any i, we can know if the eye is right or left. But here, we have two left and two right eyes, eyes[0] and eyes[3] are the left ones, and eyes[1] and eyes[2] are the right ones.

And this is my question, now if i have to get only one of the two detected results for both right and left eye, how do i do that?

Earlier i thought that eye[i][1] could be used as some kind of score. I didn't have access to the Internet for a while so I assumed that. But its not some quality score, as i just discovered.

So in our case, eyes[0][1] is 13, and eyes[1][1] is 4, eyes[2][1] is 4 and eyes[3][1] is 6. Lets call this Q

In the rest of the detections i saw a pattern that the Q for my other detections was high, usually around 40-60 so i thought maybe Q within that range denotes best detection.But as before i said, its not.

So how i do determine which 'eye' in 'eyes' is the best detection?

Answer 1

I would say that you can use previous results and give more points for the detected eyes that are present for each image (assuming you're working using the video).

You can also use parameters, such as the distance inter eye to find the best matching couple. You can finally also go to something a bit more complex (like coupled head/eye detection or background substraction) to get a pattern of yourself and search for the best matchgin eye position.

If the method gives you 4 eyes, you either have to enhance its results before by preprocessing the image or put some more complexity in your algorithm to filter out results.