How the Chain Rule Works

Composition means plugging one function into another. Here g(x) = 2x + 1 runs first, then its output feeds into f(g) = g². The value flows left to right: x = 1.5 → g = 4.0 → f = 16.00.

Each function has a local derivative -- its own rate of change. For g(x) = 2x + 1, dg/dx = 2 (always 2). For f(g) = g², df/dg = 2g = 8.0 (depends on what g outputs).

The chain rule says: multiply the local derivatives together. At x = 1.5: g'(x) = 2 × f'(g) = 8.0 = 16.0. A tiny nudge to x gets amplified 16.0× at the output.

With three functions composed as f(g(h(x))), the chain rule still works the same way -- just multiply all the local derivatives together. Notice how dh/dx = cos(1.0) = 0.540 can shrink or flip the total derivative depending on x.

Each layer in a neural network is just a function. The forward pass goes left to right, and backpropagation sends gradients right to left. At each layer, multiply by the local derivative -- that's the chain rule in action.

In a shallow network, the gradient passes through few multiplications and stays strong. In a deep network, the chain rule multiplies many local derivatives together. If those derivatives are small (< 1), the gradient vanishes -- early layers barely learn. Understanding the chain rule led to innovations like ResNets, batch normalization, and careful initialization that keep gradients flowing.