Ensemble learning
lastmod 2026-09-14created 2026-09-141 edits
In statistics and machine learning, an ensemble learning method uses multiple learning algorithms to obtain better predictive performance than could be obtained from any of the learning algorithms alone. Methods are divided into Bagging, Boosting, and stacking. Unlike a statistical ensemble in statistical mechanics, an ensemble in machine learning refers to a concrete finite set of alternative models, but it typically allows a much more flexible structure among those alternative models.
About
The main techniques of ensemble learning are voting, Bagging, Boosting, and Stacking/Blending. These techniques induce high diversity among the base models, raising the variability of the composite model. Bagging is short for "bootstrap aggregating"; it creates diversity by generating random samples from the training data and fitting the same model to each different sample. Boosting follows an iterative process that sequentially trains the next base model by weighting the errors of the previous base model, aiming to reduce the final model error. Stacking combines various independently trained base models to produce a composite model.