Overfitting
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A failure mode where a machine learning model memorizes noise in its training data, scoring well in training but failing on data it has never seen.
What is Overfitting?
Overfitting is a failure mode where a machine learning model learns the noise and quirks of its training data instead of the underlying pattern, so it scores well in training but fails on new data. The model has memorized its examples rather than generalized from them.
Training accuracy alone will not reveal it. The model that looks flawless on the data it was trained on is sometimes the one least ready for deployment.
How Overfitting Happens
Three conditions drive it. A model with more capacity than the problem needs – a deep decision tree, a large neural network – has enough room to encode individual training examples. Too little training data gives random fluctuations the statistical weight of real signal.
The third condition is feature clutter. Irrelevant or redundant input features hand the model patterns that exist only in that particular sample, and it fits them anyway.
Overfitting vs Underfitting
An overfit model has learned too much from its training data; an underfit model has learned too little. Both fail on new data, but the diagnosis differs: overfitting shows a wide gap between training and validation performance, while underfitting performs badly on both.
| Criterion | Overfitting | Underfitting |
|---|---|---|
| Root cause | Model too complex for the data | Model too simple for the pattern |
| Training error | Low | High |
| Error on new data | High | High |
| Typical fix | Regularization, more data, early stopping | More capacity, better features, less regularization |
Preventing Overfitting
- Regularization adds a penalty for complexity to the training objective, so the model keeps only the patterns strong enough to earn their cost.
- Early stopping halts training the moment performance on held-out data stops improving.
- Cross-validation rotates the held-out slice of data, exposing a model that only works on one lucky split.
- More or better data, including data augmentation, shrinks the influence of any single noisy example.
Example of Overfitting
Train a decision tree to approve loan applications using 200 past cases and every field on the form. Left unrestricted, the tree keeps splitting until each leaf covers a handful of applicants, and it classifies all 200 training cases perfectly.
Then new applications arrive, and accuracy collapses. The splits never captured creditworthiness; they encoded the individual people in the training file, down to quirks like a specific ZIP code paired with a specific income. Limiting the tree’s depth costs a few training errors and buys a model that works on applicants it has never seen.
Related AI terms: Underfitting · Bias-Variance Tradeoff · Regularization · Early Stopping · Cross-Validation
Did you like the Overfitting gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About