Learn AISend the error backward.
Trace a mistake through two weights, calculate both gradients, and then update them together.
By the end: Distinguish backpropagation from optimization and verify a simultaneous two-weight update.
Start with something concrete.
Imagine two volume knobs in a row. The first changes a signal, and the second changes the result. To fix the final output, you need to know how sensitive it is to each knob. A computational graph records that chain of dependencies.
Try the interactive model ↓Backpropagation has calculated gradients of −8 and −24. Have the weights changed yet?
Change one thing. Watch what follows.
- Start at w₁ = 3 and w₂ = 1. Follow the animated forward and backward passes.
- Reveal the gradients before updating. Notice that the prediction is still 6.
- Apply one update at rate 0.01. Confirm w₁ = 3.08 and w₂ = 1.24, then reset and try different weights.
Loading this browser experiment…
First, keep the forward values
We compute h = w₁x, prediction = w₂h, and L = ½(prediction − target)². The factor ½ makes the derivative simpler; it is a different scaling from the previous lesson, so compare losses within this example.
With input 2, target 10, and weights 3 and 1, hidden value h = 6, prediction = 6, and loss = 8. Keep these intermediate values: the backward pass will reuse them.
Multiply local sensitivities along each path
The derivative of loss with respect to prediction is prediction − target, here −4. Changing w₂ changes prediction by h times as much, so its gradient is −4 × 6 = −24.
For w₁, the influence passes through h and then w₂. Multiply the local sensitivities: −4 × w₂ × x = −4 × 1 × 2 = −8. This is the chain rule: changes propagate through dependent operations.
Now the optimizer can take a step
Use gradients from the same old weights for both updates. With rate 0.01, w₁ becomes 3.08 and w₂ becomes 1.24. The new prediction is 7.6384 and the loss is about 2.7886. Updating one weight before calculating the other gradient would be a different procedure.
Backpropagation reuses intermediate values rather than perturbing every parameter and rerunning the whole model. Automatic differentiation applies these rules across large graphs. Long chains can shrink or amplify gradients; initialization, residual connections, normalization, and gradient clipping can help.
A teammate updates w₂ and then uses that new w₂ to calculate the gradient for w₁. What should you change to reproduce the simultaneous update?
Take this idea with you.
The forward pass makes a prediction, the backward pass calculates gradients, and the optimizer updates parameters. Keep those responsibilities separate.