How the Chain Rule Works

Step 1 of 6: Functions Inside Functions

What happens when you compose f(g(x))?

Drag to change xx = 1.5

Composition means plugging one function into another. Here g(x) = 2x + 1 runs first, then its output feeds into f(g) = g². The value flows left to right: x = 1.5 → g = 4.0 → f = 16.00.

Drag x between −2 and 3 and follow the value through both boxes. Order matters in composition. Here f(g(x)) = (2x + 1)², but swapping the order gives g(f(x)) = 2x² + 1, a different function with a different derivative. Mathematicians write the composition as f ∘ g, read as "f after g". A neural network layer is itself a composition: a linear map, then an activation function. Stacking layers adds more links to the chain.

Step 2 of 6: Local Derivatives

Each function has its own rate of change

Change xx = 1.5

Each function has a local derivative: its own rate of change. For g(x) = 2x + 1, dg/dx = 2 (always 2). For f(g) = g², df/dg = 2g = 8.0 (depends on what g outputs).

Change x and watch which local derivative moves. g'(x) stays at 2 because g is a straight line, while f'(g) = 2g changes with g's output. That distinction is why the chain rule evaluates the outer derivative at g(x), not at x. A common mistake is to write f'(x) · g'(x). For this example that would give 2x · 2 = 4x, but the correct answer is 2(2x + 1) · 2 = 8x + 4. The two agree only at x = −1.

Step 3 of 6: Multiply the Chain

The total derivative is the product of local ones

Change xx = 1.5

The chain rule: multiply the local derivatives. At x = 1.5: g'(x) = 2 × f'(g) = 8.0 = 16.0. A small nudge to x moves the output 16.0× as much.

Move x and compare the total with the two factors. The full derivative here is df/dx = 4(2x + 1), so at x = 1.5 it is 16, and at x = −0.5 it is zero because g = 0 sits at the bottom of the parabola. Leibniz notation makes the rule look like fraction cancellation: df/dx = (df/dg)·(dg/dx). That is a good memory aid, though derivatives are limits, not fractions, so the cancellation is a mnemonic and not a proof.

Step 4 of 6: A Longer Chain

What about f(g(h(x)))?

Change xx = 1.0

With three functions composed as f(g(h(x))), the rule is the same: multiply all three local derivatives. Here dh/dx = cos(1.0) = 0.540 can shrink the total or flip its sign, depending on x.

Slide x from −3 to 3 and watch the cos(x) factor. At x = ±π/2 (about ±1.57) cos(x) is zero, so the whole product is zero no matter what the other links do. One zero factor anywhere in a chain kills the gradient, which is what happens with dead ReLU units. For n functions the rule extends directly: d/dx f₁(f₂(⋯fₙ(x))) is the product of n local derivatives, each evaluated at its own input.

Step 5 of 6: Chains in Networks

Each layer is a function — backprop IS the chain rule

Each layer in a neural network is a function. The forward pass goes left to right, and backpropagation sends gradients right to left. Each layer multiplies the incoming gradient by its local derivative. With several neurons per layer, each local derivative is really a matrix of partial derivatives (a Jacobian), so the products are matrix products. For one input and one output, that works out to a sum over every path between them of the product of local derivatives along the path.

Watch the animation: dots run forward through the 3-4-3-2 network, then gradients run back. With several nodes per layer, each local derivative becomes a Jacobian matrix of partial derivatives, and the product of numbers becomes a product of matrices. Backpropagation evaluates that product from the output end, multiplying a vector by one Jacobian at a time. This reverse order costs about as much as one forward pass, whether the network has a hundred weights or a billion.

Step 6 of 6: Why It Matters

Without the chain rule, deep learning wouldn't work

Here each of six layers multiplies the gradient by 0.4, so the input end receives 0.4⁶ ≈ 0.004 of it. When local derivatives are below 1 across many layers, the gradient vanishes and early layers barely learn. When they are above 1, it can explode instead. Fixes such as ResNets, batch normalization, and careful initialization keep gradients flowing.

Compare the bar heights from Output to Input. With a factor of 0.4 per layer, ten layers would leave 0.4¹⁰ ≈ 0.0001 of the gradient. Sigmoid makes this worse: its derivative is at most 0.25, so deep sigmoid networks were hard to train before about 2010. Factors above 1 cause the opposite problem, exploding gradients. ResNets add skip connections whose derivative includes an identity term, so the gradient always has a path with factor 1.

What is the chain rule?

The chain rule gives the derivative of a composed function: if y = f(g(x)), then dy/dx = f'(g(x)) · g'(x). In words, the rate of change of the whole chain is the product of the rates of change of its links, each evaluated at the value flowing through it.

Every neural network is a long composition of simple functions, so the chain rule is how its gradients are computed. Backpropagation is the chain rule applied from the loss back to every weight, and automatic differentiation in PyTorch, JAX, and TensorFlow is a general implementation of it. It also explains vanishing and exploding gradients, which shaped architectures such as ResNets and LSTMs.

Key ideas

The derivative of f(g(x)) is f'(g(x)) · g'(x).
The outer derivative is evaluated at the inner function's output, not at x.
A chain of any length differentiates to a product of local derivatives.
For vector-valued layers, the local derivatives are Jacobians and the product is matrix multiplication.
Backpropagation is the chain rule evaluated from the output backward.
Long products of factors below 1 make gradients vanish; factors above 1 make them explode.

Common questions

What is the multivariable chain rule?

When a variable affects the output through several paths, you add the contributions of each path. If z depends on x and y, and both depend on t, then dz/dt = (∂z/∂x)(dx/dt) + (∂z/∂y)(dy/dt). In a network, a weight that feeds several neurons gets its gradient by summing over those neurons.

How is the chain rule different from the product rule?

The product rule handles two functions multiplied together: (f·g)' = f'·g + f·g'. The chain rule handles one function fed into another: (f∘g)' = (f'∘g)·g'. Many expressions need both, for example x·sin(x²).

Why does backpropagation go backward instead of forward?

A network has one scalar loss and many parameters. Going backward computes the gradient of that one output with respect to all inputs in a single sweep. Going forward would need one sweep per parameter, so reverse mode is far cheaper whenever outputs are few and inputs are many.

Who discovered the chain rule?

Leibniz used it in the 1670s, and it appears in the early calculus texts of the following decades. Its use for training multi-layer networks was popularized by Rumelhart, Hinton, and Williams in 1986, building on earlier work on reverse-mode differentiation by Linnainmaa and Werbos.

Related explainers

Derivatives — The calculus behind gradient descent
Partial Derivatives & Gradients — Extend single-variable derivatives to functions of multiple variables and gradient vectors
Matrix Calculus & Jacobians — Partial derivatives, gradients, and the Jacobian — calculus for vectors
Backpropagation — How networks learn from their mistakes
Vanishing & Exploding Gradients — Why deep networks struggle and how to fix it
Skip Connections & ResNets — Residual learning and why deeper networks work

References

Last updated October 25, 2026