Backpropagation
The algorithm that trains neural networks by propagating the error from the output layer backwards through the network, computing how much each weight contributed to the mistake.
Backpropagation, short for “backward propagation of errors,” is the workhorse algorithm behind modern deep learning. During training, a network makes a prediction, a loss function measures how wrong it was, and backpropagation calculates the gradient of that loss with respect to every weight in the network.
It works in two passes:
- Forward pass: inputs flow through the layers to produce an output and a loss.
- Backward pass: using the chain rule of calculus, the error is sent back layer by layer, accumulating the contribution of each weight.
Those gradients then feed an optimiser — usually a variant of gradient descent — which nudges every weight slightly in the direction that reduces the error. Repeat across millions of examples and the network improves.
Backpropagation’s efficiency is what made deep networks practical: it computes all gradients in roughly the same cost as one forward pass, rather than perturbing each of billions of parameters individually. Popularised for neural networks in the 1980s, it remains essentially unchanged at the core of training today’s largest models.