Mini-Batch
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A small subset of training data, typically 32 to 512 examples, that a model processes together for a single parameter update during training.
What is a Mini-Batch?
A mini-batch is a small subset of the training dataset, typically 32 to 512 examples, that a model processes together to compute one parameter update. It is the middle ground between full-batch training (the whole dataset per update) and pure stochastic training (one example per update), and it is how nearly all deep learning models are trained.
How Mini-Batches Work
Each epoch, the shuffled dataset is cut into batches. Every batch produces one iteration: compute the loss on those examples, derive the gradient, update the weights. Iterations per epoch equals dataset size divided by batch size.
The batch size earns its keep on hardware. A GPU processes all examples in a batch in parallel, so a batch of 256 costs little more time than a batch of 1; the ceiling is GPU memory, which must hold the batch and its intermediate activations.
Batch size is also a hyperparameter with statistical consequences. Small batches give noisy gradients that help stochastic gradient descent escape poor minima; large batches give smoother, more accurate gradients but sometimes generalize worse.
Example of a Mini-Batch
Training an image classifier on CIFAR-10’s 50,000 images with a batch size of 128 splits each epoch into 391 iterations. At every iteration the GPU loads 128 images, runs them through the network at once, averages their errors into one gradient, and nudges the weights.
Raise the batch size to 512 and each epoch shrinks to 98 larger updates; drop it to 32 and it stretches to 1,563 noisier ones. Same data, three different training dynamics.
Related AI terms: Epoch · Stochastic Gradient Descent · Gradient Descent · Batch Normalization · GPU Memory
Did you like the Mini-Batch gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About