Ensemble Learning
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A machine learning approach that combines multiple models so their individual errors cancel, via three main strategies: bagging, boosting, and stacking.
What is Ensemble Learning?
Ensemble learning is a machine learning approach that combines the predictions of multiple models so their individual errors cancel, producing a combined predictor more accurate than any single member. The same idea is also called an ensemble method; the two terms are used interchangeably.
The logic is the wisdom of crowds applied to models: as long as the members are reasonably good and make different mistakes, aggregating them washes the mistakes out.
Types of Ensemble Learning
- Bagging trains each model independently on a random bootstrap sample of the data, then averages or votes. Random forest is the standard example: hundreds of decision trees, one vote each. Bagging mainly reduces variance.
- Boosting trains models in sequence, each one focused on the examples its predecessors got wrong. Gradient boosting and its fast implementation XGBoost are the standard examples. Boosting mainly reduces bias.
- Stacking feeds the predictions of several different models into a meta-model, which learns how to weigh and combine them into a final answer.
Bagging and boosting attack the two halves of the bias-variance tradeoff from opposite ends, which is why the choice between them depends on how the base model fails.
Example of Ensemble Learning
The Netflix Prize, a public competition to improve the accuracy of Netflix’s movie-rating predictions by 10%, was won in 2009 by a team whose solution was an ensemble: a blend of hundreds of individual models, each weak in a different way, combined into one predictor that no single model could match.
The same pattern runs a spam filter. One classifier reads word frequencies, another sender reputation, a third message structure. Each alone misfires on some emails; their combined vote catches spam that any single model would let through, with fewer false alarms on legitimate mail.
Related AI terms: Ensemble Method · Random Forest · XGBoost · Decision Tree · Bias-Variance Tradeoff
Did you like the Ensemble Learning gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About