Selecting Best Features in a feature vector using Adaboost

Question

I've read some documentation on how Adaboost works but have some questions regarding it.

I've also read that Adaboost also picks best features from data apart from weighting weak classifiers to and use them in testing phase to perform classification efficiently.

How does Adaboost pick best features from the data?

Correct me if my understanding of Adaboost is wrong!

Answer 1

In some cases the weak classifiers in Adaboost are (almost) equal to features. In other words, using a single feature to classify can result in slightly better than random performance, so it can be used as a weak classifier. Adaboost will find the set of best weak classifiers given the training data, so if the weak classifiers are equal to features then you will have an indication of the most useful features.

An example of weak classifiers resembling features are decision stumps.

Answer 2

OK, adaboost selects features based on its basic learner, tree. For a single tree, there are several means to estimate how much contribution a single feature does to the tree, called relative importance somewhere. For adaboosting, an ensamble method, containing several such trees, the relative significance of each feature to the final model can be calculated by measuring significance of each feature to each tree then average it.

Hope this can help you.