Effects of boosting with strong classifier

Question

What is the effect of boosting with strong (instead of weak, error rate close to random) classifier? Could it be possible that a strong classifier perform better by itself than when this strong classifier is used in adaboost along with a bunch of weak classifiers?

Answer 1

Yes, it is possible. All depends of your learning dataset. Look at the no free lunch theorem, there is always dataset that don't fit a particular algorithm / heuristic (even combination of thoses).

Things got more interesting with boosting when you use algorithms within the same error rate, on differents dataset. The fact that classifier should be strong or weak doesn't change the benefit of boosting. But the theorem in the foundation of the boosting specified that it inferior limit is bunch of weak classifier. If you use less than weak classifier, it won't work.

In my experiences, I never found a problem where I found a so good/strong classifier that any other classifiers(better than random) doesn't improve performance with boosting on some dataset.