Statistical Machine Learning, Spring, 2012

Materials: The main materials for the course will be lecture notes, along with supplementary readings from The Elements of Statistical Learning and Convex Optimization.

Course Final: Thursday, May 24, 10am-12pm. 74-1069. (GCCIS PhD program conference room.) You can use the course notes, the course textbook, and any handwritten notes. You cannot use any other outside materials, calculators, cell phones, etc.

Notes:
1) Background
2) Overfitting, Model Selection, Cross Validation, Bias-Variance + EoSL 7.2, 7.3, 7.4, 7.10
3) Empirical Risk Minimization and Optimization + Convex Optimization 9.1-9.5
4) Linear Methods + EoSL, Chapters 3 and 4 except 4.3. (No Linear Discriminant Analysis.)
5) Template Methods + EoSL, 2.3, 2.4, 2.5
6) Basis Expansions + EoSL, 5.1-5.3
7) Lagrange Multipliers + Convex Optimization, Ch. 5
8) Kernel Methods + EoSL, Ch. 12
9) Automatic Differentiation and Neural Networks + EoSL Ch. 11
10) Probabilistic Learning + Statistical Modeling: The Two Cultures
11) Expectation Maximization
12) Trees

Suggested References for Assumed Background:
Probability Theory
Linear Algebra and Matrix Calculus
Matrix Cookbook
Matrix Identities