Cross-Entropy Loss
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A classification loss that measures how far a model's predicted probabilities fall from the true labels. Confident wrong predictions cost the most.
What is Cross-Entropy Loss?
Cross-entropy loss (also called log loss) measures how far a classification model’s predicted probabilities fall from the true labels. Its defining trait is how it treats confidence: predicting 1% for the correct class costs vastly more than predicting 40%.
It is the standard training loss for classifiers, from logistic regression to deep networks. For regression tasks, MSE and MAE play the same role that cross-entropy plays for classification.
How Cross-Entropy Loss Works
For a single prediction over \(n\) classes, the loss is:
\[ L = - \sum_{i=1}^{n} y_i \log(p_i) \]where \(y_i\) is 1 for the true class and 0 otherwise, and \(p_i\) is the probability the model assigned to class \(i\). Only the probability given to the true class matters, and the loss is its negative logarithm.
The logarithm is what makes the loss punish overconfidence. As the probability assigned to the true class approaches zero, \(-\log(p)\) grows without bound, so the model gets a far stronger correction for a confident miss than for a hesitant one. In neural networks, a softmax layer first converts raw scores into the probabilities the loss consumes.
Example of Cross-Entropy Loss
Take a spam filter classifying one email whose true label is spam. If the model predicts spam with probability 0.9, the loss is \(-\ln(0.9) \approx 0.11\). At probability 0.5 the loss rises to about 0.69, and at 0.01, a confident wrong call, it jumps to about 4.6.
Training averages this loss over every example and adjusts the weights to reduce it. Each step pushes predicted probabilities toward the true labels, and pushes hardest on the examples the model gets confidently wrong.
Related AI terms: MSE · MAE · Classification · Softmax · Logistic Regression
Did you like the Cross-Entropy Loss gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About