Comparision with grund truth image

Question

I am currently working on object detection.I have obtained the co-ordinates of the object in the image.I am also having the ground truth (original) co-ordinates of the objects in the image.Now i want to compare the obtained co-ordinates with the original one.So is there any method for doing that..or any method for calculating detection accuracy..thanks in advance..

Answer 1

There is multiple ways to achieve that depending on your needs, but in our case we use the Modified Hausdorff Distance. You can find a working matlab code on filexchange here. It is fairly easy to use. The function takes the 2 point sets and returns a distance.

Another method, faster but a less robust is the quotient of similarity defined as 2*pixels included in both object, over the addition of the total number of pixels in each object.

2*N(a,b) / ( N(a)+N(b) )

where N is the number of pixels.

EDIT for graphical comparison:

We use the fileexchange rendering function sc. You can easily create a mask with 2 binary images. You can create a new image by adding your objects over your ground truth. You can apply the mask with a different color, so the differences will stand out.

Answer 2

People in object detection field (see Pedro Felzenszwalb page) typically use bounding box overlap to measure detection accuracy (position is not everything since there is also an extent). The formula is

overlap = area(bbox1 & bbox2)/area(bbox1 | bbox2)

where & means intersection and | means union. Note that this doesn't require mapping pixels in two objects (which may not be well defined) as another answer suggested. Bounding box can be easily calculated if its axes are aligned with the coordinate system as min-max coordinates. For rotated bounding boxes you can fit an ellipse and use rotatedRect class:

RotatedRect rect;
vector<Point2f> pts; // you have to fill out pts
...
rect = fitEllipse(pts);

Finally if you need to know if a given point is inside of a polygon you can use a pointPolygonTest.