Regularization
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
Techniques that penalize model complexity during training, trading a slightly worse fit on training data for better performance on new data.
What is Regularization?
Regularization is any technique that penalizes a model’s complexity during training, accepting a slightly worse fit on the training data in exchange for better performance on new data. It is the standard remedy for overfitting.
The core idea: a pattern must be strong enough to justify the complexity it adds. Weak patterns, which are usually noise, get suppressed.
Types of Regularization
- L1 regularization (lasso) penalizes the absolute size of coefficients. It pushes weak coefficients to exactly zero, so it doubles as feature selection – the model tells you which inputs it dropped.
- L2 regularization (ridge) penalizes squared coefficients. It shrinks all weights toward zero without eliminating them, spreading influence across correlated features instead of betting on one.
- Elastic net combines L1 and L2, useful when the data holds groups of correlated features and you want both shrinkage and selection.
- Deep learning variants: dropout randomly disables neurons during training so no single pathway becomes indispensable, and weight decay applies the L2 idea to network weights. Early stopping acts as regularization too, by cutting off training before the model fits noise.
Example of Regularization
A hospital builds a linear model to predict readmission risk from 300 patient variables, using 2,000 patient records. With that many inputs and that little data, an unregularized model assigns confident weights to spurious correlations, and its predictions fall apart on the next year’s patients.
Adding an L1 penalty changes the outcome step by step. During training, every coefficient now has to overcome the penalty to stay nonzero; roughly 260 of the 300 shrink to zero. The surviving 40 variables carry genuine signal, training accuracy drops slightly, and accuracy on new patients rises. The sparser model is also easier for clinicians to inspect, since it names the factors it actually uses.
Related AI terms: Overfitting · Early Stopping · Dropout · Weight Decay · Bias-Variance Tradeoff
Did you like the Regularization gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About