AdaBoost
lastmod 2024-04-24created 2015-10-082 edits
AdaBoost (short for adaptive boosting) is a machine learning meta-algorithm developed by Yoav Freund and Robert Schapire, who received the 2003 Gödel Prize for developing it. AdaBoost can be used in combination with many other types of learning algorithms to improve performance.
The final output of the boosted classifier can be expressed as a weighted sum of the outputs of other learning algorithms (weak learners).
AdaBoost is adaptive in the sense that subsequent weak learners can correct the instances misclassified by previous classifiers. As a result, AdaBoost is sensitive to noisy data and outliers. In some cases, however, it can be less prone to overfitting than other learning algorithms. Even if the individual learners perform poorly, as long as each one performs slightly better than random guessing (an error rate below 0.5 in binary classification), it can be proven that the final model converges to a strong learner.