Backpropagation
Backpropagation applies the chain rule to compute the gradient of the loss with respect to parameters layer by layer from output to input; it is the core of neural network training.
Purpose
To compute neural network gradients efficiently, making deep learning training feasible.
Mechanism
Forward propagation computes each layer's activations. Backpropagation starts from the loss and propagates gradients layer by layer with the chain rule: ∂L/∂w = ∂L/∂y * ∂y/∂w. Matrix multiplication enables efficient parallel computation. Its complexity matches the forward pass.
Use cases
- Neural network training
- Deep learning
- Gradient computation