Lesson 9 of 21 · Part III, Validation and theory

Validation (part 1) — model selection and assessment

Why estimates go wrong: model selection versus model assessment, the random-target counterexample, grid and random search, K-fold CV, sampling and error measures.

4 figures to manipulate.

In this lesson

• Preliminaries: bias and variance
• Motivations
• The two goals of validation
• An instructive counterexample
• Grid search
• K-fold cross-validation
• Special cases
• Error measures for the evaluation

Other lessons

Neural networks (part 2) — training in practice · Validation (part 2) — formal schemes · All lessons

Notes by Fabio Piscitelli — Machine Learning (654AA), Prof. Alessio Micheli, University of Pisa, a.y. 2026/27.