Lesson 10 of 21 · Part III, Validation and theory

Validation (part 2) — formal schemes

Risk and empirical risk, then the rigorous schemes: model selection and model assessment with hold-out and K-fold CV, and how to combine them (TR–VL–TS, CV + test, double CV).

5 figures to manipulate.

In this lesson

• Notation
• Methodology: identifying the goal
• Model selection
• Risk estimation (model assessment)
• Combining model selection and model assessment

Other lessons

Validation (part 1) — model selection and assessment · Validation (part 3) — typical mistakes and FAQ · All lessons

Notes by Fabio Piscitelli — Machine Learning (654AA), Prof. Alessio Micheli, University of Pisa, a.y. 2026/27.