Model Evaluation
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
The process of measuring how well a trained model performs on data it has never seen, with metrics matched to the task and the cost of each error.
What is Model Evaluation?
Model evaluation is the process of measuring how well a trained machine learning model performs on data it has never seen, using metrics matched to the task and to the cost of each kind of error.
The phrase “never seen” carries the weight. A model scored on its own training data reports how well it memorized, not how well it generalizes; evaluation exists to catch overfitting before deployment does.
How Model Evaluation Works
Performance is measured on a held-out test set whose correct labels (ground truth) the model never trained on. When data is scarce, cross-validation rotates the held-out portion so every example gets tested once.
The metric depends on the task. For classification, the confusion matrix yields accuracy, precision, recall, the F1 score, and the ROC curve with its AUC. For regression, MSE and MAE measure how far predictions land from actual values.
One aggregate number is rarely enough. A model that scores well overall still fails badly for a particular subgroup, region, or age band without the headline metric moving, which is why careful evaluation reports results per slice and treats fairness as something to measure, not assume.
Example of Model Evaluation
A bank tests its fraud model on 100,000 transactions, 100 of them fraudulent. The model predicts “legitimate” for nearly everything and posts 99.9% accuracy.
The confusion matrix tells the real story: 0 of the 100 frauds were caught. Recall is 0, F1 is 0, and the accuracy figure was earned entirely on the 99,900 easy negatives.
This is the accuracy paradox, and it is the standard argument for evaluating with precision, recall, and F1 whenever classes are imbalanced. The right metric is the one aligned with the error you cannot afford.
Related AI terms: Confusion Matrix · F1 Score · Ground Truth · Test Set · Cross-Validation
Did you like the Model Evaluation gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About