Dropout
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A regularization technique that randomly switches off a fraction of a network's neurons at each training step so no single neuron becomes indispensable.
What is Dropout?
Dropout is a regularization technique that randomly deactivates a fraction of a neural network’s neurons at each training step, which prevents overfitting by stopping the network from depending on any single neuron.
The idea, introduced by Nitish Srivastava, Geoffrey Hinton, and colleagues in a 2014 paper, is counterintuitive: you make the network better by repeatedly breaking random pieces of it.
How Dropout Works
Before each training batch, every neuron in a dropout layer is switched off with a fixed probability, the dropout rate. A rate of 0.5 silences a random half of the layer. Silenced neurons take no part in that batch’s forward pass or weight updates.
Because the set of silenced neurons changes every batch, the network never trains the same sub-network twice. Each pass trains a different “thinned” version of the model, and the full network at test time behaves like an average of them all – an ensemble for the price of one network.
At inference, dropout is turned off. Every neuron participates, with outputs scaled to match the activity level seen during training.
Example of Dropout
Take a network for reading handwritten digits with a 128-neuron hidden layer and a dropout rate of 0.5. On the first batch, a random 64 neurons go silent; on the next batch, a different random 64.
Without dropout, one neuron might grow into the sole detector for the loop of a “6,” and the rest of the network would lean on it. With dropout, that neuron vanishes half the time, so other neurons must learn to spot the loop too. The knowledge spreads out instead of concentrating.
When training ends, all 128 neurons switch on for good. The network now holds several redundant paths to the right answer, and its accuracy on new handwriting is higher than the un-regularized version – the whole point of the exercise.
Related AI terms: Regularization · Overfitting · Label Smoothing · Neural Network · Model Training
Did you like the Dropout gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About