Boosting (machine learning)
lastmod 2026-08-03created 2026-08-032 edits
In machine learning, boosting is an ensemble meta-algorithm mainly for reducing bias and variance in supervised learning, and a family of machine learning algorithms that convert weak learners into strong learners. Boosting is based on the question posed by Kearns and Valiant (1988, 1989): "Can a set of weak learners create a single strong learner?" A weak learner is defined as a classifier that is only slightly correlated with the true classification (it can label examples better than random guessing). In contrast, a strong learner is a classifier that is arbitrarily well-correlated with the true classification.
Robert Schapire's affirmative answer to the Kearns and Valiant question in a 1990 paper had significant ramifications in machine learning and statistics, most notably leading to the development of boosting.
When first introduced, the hypothesis boosting problem simply referred to the process of turning a weak learner into a strong learner.
See also
- Ensemble learning
- Korean: 랜덤 포레스트
- Bagging
- Logistic regression
- Principle of maximum entropy
- Cross-validation
- Korean: 그라디언트 부스팅 (Gradient boosting)