Partial Derivatives & Gradients
From 1D to 2D
Single derivatives become partial derivatives in multiple dimensions
In 1D, f(x) = x² has one derivative: df/dx = 2x. There's only one direction to change, so one slope tells the whole story.
Step 1 of 6: From 1D to 2D
Single derivatives become partial derivatives in multiple dimensions
In 1D, f(x) = x² has one derivative: df/dx = 2x. There's only one direction to change, so one slope tells the whole story.
Step 2 of 6: Partial Derivative Notation
Hold one variable fixed, differentiate the other
∂f/∂x = 4.00 — the function is increasing in x at this point. The teal tangent line tilts upward. We hold y = 1 fixed and only vary x.
Step 3 of 6: The Gradient Vector
Combine all partial derivatives into one vector: ∇f
Strong gradient! Far from the minimum, the surface is steep. The gradient arrow grows because the bowl gets steeper as you move outward.
Step 4 of 6: Gradient Direction & Magnitude
The gradient points uphill with magnitude equal to the steepness
Strong gradient (|∇f| = 0.84) pointing at -87°. The surface is steep here. Moving opposite to this direction is the fastest way downhill — that's gradient descent.
Step 5 of 6: Gradient Descent Connection
Step opposite to ∇f to minimize the loss
Starting point (2.5, 2.0) — far from the minimum. The teal arrow shows −∇f, the direction gradient descent will step. Increase the step slider to descend.
Step 6 of 6: Higher Dimensions
The same idea scales to millions of parameters
In 2D, the gradient is a vector with 2 components — one partial derivative per variable. We can still visualize this. In ML, the concept is identical, just with more dimensions.