What Softmax Meaning, Applications & Example
Function that converts numbers into probabilities.
What is Softmax?
The Softmax function is a mathematical function used in machine learning, particularly for multi-class classification problems. It converts a vector of raw scores (logits) into probabilities, where the sum of the probabilities is 1. This makes it useful for predicting the likelihood of each class in a classification task. The Softmax function is applied to the output of a model , and it ensures that the model’s predictions are interpretable as probabilities.
Softmax Formula
The formula for the softmax function is:
\[ \text{Softmax}(z_i) = \frac{e^{z_i}}{\sum_{j} e^{z_j}} \]Where:
- \( z_i \) is the score for class \( i \).
- The denominator is the sum of the exponential of all scores, ensuring that the sum of the probabilities is 1.
Applications of Softmax
- Multi-class Classification: Used in models where there are more than two possible classes, such as image classification with multiple categories.
- Neural Networks: Typically used in the output layer of neural networks for multi-class classification tasks.
Example of Softmax
Given a set of scores \([2.0, 1.0, 0.1]\), the Softmax function would convert them into probabilities.
For each score \( z_i \):
\[ \text{Softmax}(z_1) = \frac{e^{2.0}}{e^{2.0} + e^{1.0} + e^{0.1}} \approx 0.659 \]\[ \text{Softmax}(z_2) = \frac{e^{1.0}}{e^{2.0} + e^{1.0} + e^{0.1}} \approx 0.242 \]\[ \text{Softmax}(z_3) = \frac{e^{0.1}}{e^{2.0} + e^{1.0} + e^{0.1}} \approx 0.099 \]Thus, the probabilities are approximately \([0.659, 0.242, 0.099]\). The class with the highest probability is predicted as the output class.