Chain Rule
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
The calculus rule for differentiating composite functions; in machine learning, the engine of backpropagation, tracing error back through every layer.
What is the Chain Rule?
The chain rule is the calculus rule for differentiating a function built from other functions: the derivative of the whole is the product of the derivatives of its parts.
A neural network is exactly such a function, layer composed on layer, which makes the chain rule the mathematical engine of backpropagation and, with it, of deep learning.
How the Chain Rule Works
If \( z \) depends on \( y \), and \( y \) depends on \( x \), the chain rule links them:
\[ \frac{dz}{dx} = \frac{dz}{dy} \cdot \frac{dy}{dx} \]Read it as sensitivities multiplying along a path. If a change in \( x \) doubles \( y \), and a change in \( y \) triples \( z \), then a change in \( x \) multiplies through to \( z \) six-fold. The rule extends to any depth of nesting: differentiate each link, multiply along the chain.
The Chain Rule in Backpropagation
Training needs the gradient \( \frac{\partial L}{\partial w} \): how much the loss \( L \) would change if a single weight \( w \) changed. Between that weight and the loss sit many layers of computation, so the gradient is a chain of local derivatives multiplied together, from the loss back through each layer to the weight.
Backpropagation is the algorithm that applies this systematically, sweeping backward through the network once and producing the gradient for all the weights together. Gradient descent then uses those gradients to update the weights.
The multiplication also explains a famous failure mode. When the local derivatives are small, their product shrinks toward zero as the chain gets longer, so early layers of deep networks stop learning. That is the vanishing gradient problem.
Example of the Chain Rule
Start with pure math: let \( z = y^2 \) and \( y = 3x \). Then \( \frac{dz}{dy} = 2y \) and \( \frac{dy}{dx} = 3 \), so \( \frac{dz}{dx} = 2y \cdot 3 = 18x \). Two easy derivatives, multiplied, give the derivative of the composite.
Now the same idea inside a spam classifier. To know how one weight deep in the network affects the loss, backpropagation multiplies three local sensitivities: how the loss changes with the prediction, how the prediction changes with that neuron’s output, and how the neuron’s output changes with the weight.
The product tells the optimizer exactly which way, and how hard, to nudge that weight. Repeated for every weight over millions of examples, that arithmetic is what training is.
Related AI terms: Backpropagation · Gradient Descent · Vanishing Gradient · Propagation
Did you like the Chain Rule gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About