Statistical Machine Learning, Spring, 2010

Materials: The main materials for the course will be lecture notes, along with supplementary readings from The Elements of Statistical Learning and Convex Optimization.

Lecture Notes
1) Overfitting, Cross Validation, Bias-Variance
2) Empirical Risk Minimization and Optimization
3) Linear Methods
4) Basis Expansions
5) Template Methods
6) Kernel Methods and SVMs Lagrange Duality
7) Automatic Differentiation and Neural Networks
8) Trees
9) Boosting
10) Learning Theory
11) Probabilistic Learning(Now complete. Make sure to reload.)
12) Expectation Maximization

Readings:
1) EoSL, Sections 7.2, 7.3, 7.4, 7.10
3) EoSL, Chapters 3 and 4 except 4.3. (No Linear Discriminant Analysis.)
2) Convex Optimization 9.1-9.5
4) EoSL, 5.1-5.3
5) EoSL, 2.3, 2.4, 2.5
6) EoSL, Chapter 12, Convex Optimization, Chapter 5
7) EoSL, Chapter 11
8) EoSL, 9.2
9) EoSL, Chapter 10 and Section 16.1, 16.2, 16.2.1, and 16.2.2
10) Introduction to Statistical Learning Theory, Bousquet, Boucheron, and Lugosi
11) Statistical Modeling: The Two Cultures, Breiman
12) EoSL, 8.5

Suggested References for Assumed Background:
Probability Theory
Linear Algebra and Matrix Calculus
Matrix Cookbook
Matrix Identities