XGBoost
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A fast, regularized implementation of gradient boosting that builds decision trees one after another, each trained to correct the errors of the trees before it.
What is XGBoost?
XGBoost (Extreme Gradient Boosting) is a fast, regularized implementation of gradient boosting: it builds an ensemble of decision trees in sequence, each new tree trained to correct the errors of the trees before it.
It sits under the boosting branch of ensemble learning, the counterpart to bagging methods like random forest that train their trees independently. On structured, tabular data it is a default choice and a fixture of winning entries in machine learning competitions.
How XGBoost Works
The first tree makes a rough prediction. XGBoost measures each example’s residual (how far off the prediction was) and trains the next tree to predict those residuals, using gradient descent on the loss function to decide what each tree should fix. Hundreds of rounds later, the summed corrections form the final model.
Boosting chases errors aggressively, so unchecked it memorizes noise. XGBoost builds regularization into training: penalties on tree size and leaf weights, plus pruning, hold overfitting back.
The engineering is the other half of its reputation. Parallel split-finding across CPU cores and native handling of missing values make it much faster than earlier gradient boosting tools.
Example of XGBoost
A model predicts house prices from square footage, location, and age. The first tree predicts something close to the average price for every house: cheap houses come out too high, expensive ones too low.
The second tree trains on exactly those gaps. It learns, say, that large houses in central districts were underpriced, and adds a correction. The third tree corrects what still remains, and so on for several hundred rounds, each round shaving the error smaller.
The final price estimate is the sum of all the corrections. No single tree understands houses; the sequence of error-fixers does.
Related AI terms: Ensemble Learning · Random Forest · Decision Tree · Gradient Descent · Regularization
Did you like the XGBoost gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About