Grid Search
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A hyperparameter tuning method that trains and evaluates a model on every combination of values in a predefined grid, then keeps the best-scoring one.
What is Grid Search?
Grid search is a hyperparameter tuning method that trains and evaluates a model on every combination of hyperparameter values from a predefined grid, then keeps the combination that scores best.
Its appeal is that it is exhaustive and easy to reason about: nothing in the grid goes untried. Its weakness is cost, because the number of combinations multiplies with every hyperparameter added.
How Grid Search Works
First, choose the values to test for each hyperparameter: perhaps three learning rates and four regularization strengths. The grid is every pairing of those values, twelve combinations in this case.
Then train one model per combination and score each on held-out data, with cross-validation as the standard choice. The combination with the best score wins.
The multiplication is the trap. Five hyperparameters with five values each make 3,125 combinations, and with 5-fold cross-validation that means 15,625 training runs.
Grid Search vs Random Search
The practical difference: grid search tests every combination of a fixed list of values, while random search draws configurations at random from ranges and stops when the budget runs out.
Random search is the better default for most problems. Bergstra and Bengio showed in 2012 that usually only a few hyperparameters matter for a given task, and a grid burns most of its budget re-testing identical values of the ones that don’t. Sixty random trials explore sixty distinct learning rates; a 60-point grid explores only the handful you listed.
Grid search still earns its keep when there are one or two hyperparameters, or when an exact, reproducible sweep matters more than efficiency.
Example of Grid Search
Tuning a support vector machine is the textbook case. The engineer grids the regularization parameter C over {0.1, 1, 10, 100} and the kernel over {linear, RBF, polynomial}: twelve combinations.
Each combination is trained and scored with 5-fold cross-validation, 60 training runs in total. If C = 10 with the RBF kernel posts the highest mean accuracy, that configuration is retrained on the full training data and shipped.
Related AI terms: Hyperparameter Tuning · Hyperparameter · Cross-Validation · Support Vector Machine · Optimization
Did you like the Grid Search gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About