building positive samples for rotated images for cascaded training in OpenCV

Question

I need to train a cascaded classifier to detect vehicles and different viewing angles. I'm using OpenCV.

Some of the angles that I need to capture cause the placement of the vehicle within the image to be diagonal, as shown below:

Now the problem with this is that because the vehicle is diagonally placed across the image, then there's a lot of unnecessary background which I can't crop out simply because images have to be rectangular. Is there another way to build positive samples for diagonally placed angles/perspectives of objects? I do need the classifier to be able to recognize this specific view of an object. It isn't so bad in the car above, because a car doesn't have a huge difference between its length and width. But if I do the same with a truck for example, which is several times longer than it is wide, then the images end up having more background than object of interest in them, like below. I'm worried about whether this is good or bad for classification, and if there's any solution.

Will a classifier recognize a specific angle of a vehicle regardless of which way it's rotated on the background? And in that case, would it be better to rotate the entire truck image such that the truck is horizontal/vertical, then clip the rest of the image?

Answer 1

Q: Will a classifier recognize a specific angle of a vehicle regardless of which way it's rotated on the background?

A: Nope. A classifier cannot recognize a specific angle of vehicle regardless of what way it is rotated. (Talking about Haar-like features here).There is a concept created to introduce tilted 45° Haar-like feature to improve "dimensionality" which is pretty successful from research papers I read on it in the past.

There was also an attempt to introduce generic rotated haar-like features, but it was pretty much unsuccessful from my memory. However if you are going to use super high resolution image, chances are it will work. But I won't bet money on it.

Q: To your whole problem

A: Background in positive image samples may not necessarily affect detection badly. From case to case basis, it may actually help in your detection.

My solution(if you really have to use classifiers), at least for image based detection, is to make use of OpenCV rotate() function. Where you keep rotating the image from 1° to 360°(like maybe 10° each time) and keep applying the classifiers each time. Detection time may take slightly longer, but I don't think it will be more than a few seconds.

For video wise, it will be super laggy if I am not wrong. Do give it a shot if time permits.

Another thing I wish to raise is that your vehicles(like the truck and the car), have very different features. If I were you, I would split them into different classifiers and run them at the same time for vehicle detection (Done it before with 3 different classifiers, eyes, hand and face with real time results).

If you are intending to train them into one classifier, it may or may not work, so do look into my suggestion to be on the safe side.

You may also want to look at this links: http://docs.opencv.org/modules/objdetect/doc/latent_svm.html (classifier more commonly used for objects detection. I always used haar, so have no experience in regards to this clasifier. Sorry)

http://www.araa.asn.au/acra/acra2006/papers/paper_5_63.pdf (rotation-based, contradicts my answer a little. But I stand on my case that if the image's resolution is not high enough, there maybe a lot of rounding errors.)

Hope my answer helped you. Good luck (: Do comment if you need anymore help or anything is unclear.