Lesson 8 of 21 · Part II, Linear models and neural networks

Neural networks (part 2) — training in practice

How an MLP is actually trained: initialization, on-line/batch/mini-batch, learning rate and momentum, early stopping and weight decay, Cascade Correlation, inputs and outputs, the MONK benchmark.

13 figures to manipulate.

In this lesson

• Philosophy
• Initial values of the weights
• Multiple minima
• On-line, batch and mini-batch
• The learning rate \eta
• Stopping criteria
• Overfitting and regularization
• Number of hidden units
• Input and output
• Strengths and weaknesses of neural networks
• Toward the project: the MONK benchmark

Other lessons

Notes on backpropagation · Validation (part 1) — model selection and assessment · All lessons

Notes by Fabio Piscitelli — Machine Learning (654AA), Prof. Alessio Micheli, University of Pisa, a.y. 2026/27.