The Chain Rule
Turn one dial and it drives a second stage, then a third, so the machine's response to your turn is every stage's response multiplied down the line, the exact arithmetic a network runs the moment `loss.backward()` fires.
- ▸Combining two rates, instinct says add; a chain of stages multiplies them, so two doublings quadruple, not double.
- ▸A small push at the input reaches the output scaled once for every stage it crossed.
- ▸Set one stage in a deep chain nearly flat and the whole chain's response collapses toward zero.
You already know a single machine: feed it a number, it hands one back by a fixed rule. Now bolt two of them together, the output of the first piped straight into the second. The number you enter never reaches the final readout directly. It lands first on a middle gauge as the first machine's answer, and only that middle reading enters the second machine. So the input touches the end only through what the first stage made of it. Swap which machine sits first and the same input generally lands somewhere else, order is not a formality here, it is the whole wiring.
Feed 2 into a stage that adds one and 3 shows on the middle gauge; hand that 3 to a stage that squares, and 9 comes out the end. Change the input and both the middle gauge and the final readout move, but the final readout only ever moves because the middle one did. That single fact, the input reaches the output solely through the value in the middle, is the crack this whole topic pries open.