Statistical Machine Learning, Spring, 2011

Materials: The main materials for the course will be lecture notes, along with supplementary readings from The Elements of Statistical Learning and Convex Optimization.

Notes:
1) Background
2) Overfitting, Model Selection, Cross Validation, Bias-Variance + EoSL 7.2, 7.3, 7.4, 7.10
3) Empirical Risk Minimization and Optimization + Convex Optimization 9.1-9.5
4) Linear Methods + EoSL, Chapters 3 and 4 except 4.3. (No Linear Discriminant Analysis.)
5) Template Methods + EoSL, 2.3, 2.4, 2.5
6) Basis Expansions + EoSL, 5.1-5.3
7) Kernel Methods + Lagrange Multipliers + EoSL, Ch. 12, Convex Optimization, Ch. 5
8) Automatic Differentiation and Neural Networks + EoSL Ch. 11
9) Probabilistic Learning
10) Expectation Maximization

Suggested References for Assumed Background:
Probability Theory
Linear Algebra and Matrix Calculus
Matrix Cookbook
Matrix Identities