Machine Learning 689, Spring, 2026

Course Description:

Machine learning is the computational study of artificial systems that can adapt to novel situations, discover patterns from data, and improve performance with practice. This course will cover the popular frameworks for learning, including supervised learning, reinforcement learning, and unsupervised learning. The course will provide a state-of-the-art overview of the field, emphasizing the core statistical foundations. Detailed course topics: overview of different learning frameworks such as supervised learning, reinforcement learning, and unsupervised learning; mathematical foundations of statistical estimation; maximum likelihood and maximum a posteriori (MAP) estimation; missing data and expectation maximization (EM); graphical models including mixture models, hidden-Markov models; logistic regression and generalized linear models; maximum entropy and undirected graphical models; nonparametric models including nearest neighbor methods and kernel-based methods; dimensionality reduction methods (PCA and LDA); computational learning theory and VC-dimension; reinforcement learning; state-of-the-art applications including bioinformatics, information retrieval, robotics, sensor networks and vision. Prerequisites: undergraduate level probability and statistics, linear algebra, calculus, AI; computer programming in some high level language

Preliminary list of topics:

(This list is subject to change, and feedback on topics of interest is welcome!)

• Basics (Math, loss functions, linear methods)
• Bayesian inference (Theory, probabilistic programming languages, Markov chain Monte Carlo)
• Autodiff (Scalar, matrix)
• Theory (Asymptotic rates, finite-sample bounds)
• Optimization (Deterministic, stochastic, theory)
• Neural networks (Multi-layer perceptrons, convnets, etc.)
• ELBOs and all that (Expectation-maximization, variational inference.)
• Diffusion methods
• Normalizing flows
• Gaussian processes
• Bayesian optimization
• Large language models

Should you take 589 or 689?

If in doubt, you should probably take 589. :) In the past, some students took 689 because they were not able to register for 589 and they typically regretted the decision. That said, 589 is a course on using machine learning, while 689 is a class on how to do machine learning research. The fundamental goal of this course is to teach the fundamental ideas so that you will be able to invent new methods or write research papers on machine learning. Thus, we will cover a mixture of very old ideas (like statistical asymptotics) that will always be important and some rather new ideas (like diffusion methods) that are close to the frontier of knowledge, but not much in between.

Override questions:

Please use the college system

Grading:

• Homework: 10%
• Midterm exam 1: 10-25%
• Midterm exam 2: 20-35%
• Final exam: 30-60%

Course policies:

• This course will be in-person. Lectures will not be recorded. Please do not record lectures.
• In general, the exams in this course will be very difficult. However, grading policies will be calibrated for this. That is, grading policies will reflect that a less than perfect score on the exams may still indicate excellent performance in the course. As such, grade letter boundaries may be lower than the typical boundaries.
• It is essential that you come in-person for the exams, including the final exam. Please do not schedule job interviews or person travel during the times of any of the exams, including the final exam.