Lesson 5 of 21 · Part II, Linear models and neural networks

Linear models and K-nearest neighbors

Linear regression and classification with LMS, normal equations and gradient descent, regularization; then K-NN, the Bayes classifier and the curse of dimensionality.

23 figures to manipulate.

In this lesson

• Notation for the data
• Linear models
• Classification with linear models
• The learning problem
• Limits of linear models
• Extending the linear model
• Regularization: controlling complexity
• K-nearest neighbors

Other lessons

Generalization and validation · Neural networks (part 1) — from the neuron to the MLP · All lessons

Notes by Fabio Piscitelli — Machine Learning (654AA), Prof. Alessio Micheli, University of Pisa, a.y. 2026/27.