Random Forest
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
An ensemble method that trains many decision trees on random subsets of the data and combines their votes, trading a single tree's readability for accuracy.
What is Random Forest?
Random forest is an ensemble learning method that trains many decision trees on random subsets of the data and combines their predictions: majority vote for classification, average for regression.
It sits under the bagging branch of ensemble learning, the counterpart to boosting methods like XGBoost. Where a single tree memorizes its training data, the forest’s averaging cancels each tree’s individual errors. The price is readability: no one can trace a vote of five hundred trees the way they can trace one.
How Random Forest Works
Each tree trains on a bootstrap sample: a random draw, with replacement, from the training data. This is bagging (bootstrap aggregating), and it guarantees the trees see different views of the same dataset.
At every split, each tree considers only a random subset of the features. This decorrelates the trees: without it, every tree would key on the same strong predictor and vote identically. Randomness in both data and features is what reduces overfitting.
The samples left out of each tree’s draw (the out-of-bag data) double as a built-in test set, so the forest estimates its own error without a separate holdout.
Random Forest vs Decision Tree
The practical difference: a random forest is hundreds of trees voting together, while a decision tree is one readable model. The forest wins on accuracy and stability; the single tree wins on interpretability. You trade one for the other.
| Criterion | Decision Tree | Random Forest |
|---|---|---|
| Structure | One tree | Hundreds of trees, each trained on a random data subset |
| Interpretability | Every prediction traceable as a rule path | A vote tally; not readable as rules |
| Overfitting risk | High when grown deep | Low; averaging cancels individual trees’ errors |
| Accuracy | Lower on complex data | Higher and more stable |
Example of Random Forest
A lender trains a forest of 500 trees on its loan history to predict defaults. Each tree sees a different random sample of past borrowers, and at each split a different random slice of attributes: one tree keys on income, another on payment history, a third on debt ratio.
A new application arrives. Every tree votes default or repay; 430 of 500 vote repay, and the loan is approved with that margin as a confidence signal. No single tree’s quirks decide the outcome.
When a regulator asks why the model approved it, the honest answer is a vote count, not a rule. That is the interpretability trade the forest makes, and why lenders pair it with explanation tools or a simpler model.
Related AI terms: Decision Tree · Ensemble Learning · XGBoost · Overfitting
Did you like the Random Forest gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About