Lesson 7 of 21 · Part II, Linear models and neural networks

Notes on backpropagation

The full derivation of backpropagation: the delta of each unit, the backward pass from the output to the hidden layers, the training loop and its linear cost.

5 figures to manipulate.

In this lesson

• Tools from differential calculus
• The problem
• The iterative algorithm
• The derivation
• Summary of the formulas
• The complete training cycle
• Properties and interpretation

Other lessons

Neural networks (part 1) — from the neuron to the MLP · Neural networks (part 2) — training in practice · All lessons

Notes by Fabio Piscitelli — Machine Learning (654AA), Prof. Alessio Micheli, University of Pisa, a.y. 2026/27.