University of Massachusetts Amherst
College of Information and Computer Sciences


Machine Learning

Short description:

Introduction to core machine learning models and algorithms for classification, regression, dimensionality reduction and clustering. The course will cover the mathematical foundations behind the most common machine learning algorithms, and the effective use in solving real-world applications. Requires a strong mathematical background and knowledge of one high-level programming language such as Python.

Detailed description:

This course will introduce core machine learning models and algorithms for classification, regression, clustering, and dimensionality reduction. On the theory side, the course will cover the mathematical foundations behind the most common machine learning algorithms. It will focus on understanding models and the relationships between them. On the applied side, the course will focus on effectively using machine learning methods to solve real-world problems with an emphasis on model selection, regularization, design of experiments, and presentation and interpretation of results.

The course will be held in a flipped-classroom manner, with students being assigned pre-recorded videos, and the lectures being reserved for discussions, including Q&A on the lecture topics, exercises, connecting the lecture abstractions to real-world application, implementation considerations and demos. The assignments will involve both mathematical problems and implementation tasks. Knowledge of a high-level programming language is absolutely necessary. Python is most commonly used, but languages such as Matlab, R, Scala, Julia would also be suitable. Strong foundations in linear algebra, calculus, probability and statistics are essential for the successful completion of this course.

Lectures: Monday & Wednesday 2:30-3:45pm.

Credit: 3 units


Teaching assistants:



Class materials will be posted to the Moodle course.

Discussions will happen on Piazza or over Moodle.

Course topics:

  1. Introduction to Machine Learning. Simple classifiers Reading: Bishop, Section 1.2.1-1.2.4 Probability Theory. ESL Section 2.3.2. ESL Section 2.5.

  2. Probability and estimation Reading: Bishop, 2.1 Binary Variables, 2.2 Multinomial Variables
    Advanced: Mitchell, Estimating Probabilities

  3. Naive Bayes Reading: Mitchell, 3.1 and 3.2, Naive Bayes

  4. Linear Discriminant Analysis (LDA) Reading: ESL 4.1-4.3 (p. 101-102, 106-110). Bishop 4.1.1-4.1.4 Discriminant Functions. Bishop 4.2 Probabilistic Generative Models

  5. Logistic Regression (LR) Reading: ESL Section 4.4 (p. 119-120, 127-132)
    Advanced: Mitchell, 3.3, Logistic Regression

  6. Generalization and Evaluation
    (Optional) Deep dive: Machine Learning Theory Reading: Nina Balcan, Notes on generalization guarantees.

  7. Support Vector Machines Reading: ESL Section 12.3. ESL Section 12.3.6 (p. 434-438). Bishop 6.1, 6.2 (p. 291 - 299).

  8. Ensemble Methods Reading: ESL Chapter 16 (p. 605-622). Bishop Sections 14.3,14.4 (p. 657 - 665).

  9. Linear Regression, Ridge, and Lasso Reading: ESL Sections 3.1, 3.2.1 (p. 43-51). ESL Sections 3.4.1-3.4.3 (p. 61-73).

  10. Regression trees and smoothing Reading: ESL 6.1 and 6.2 (p. 191-200). ESL 9.2.1, 9.2.2 (305-308).

  11. Neural Networks and Deep Learning Reading: ESL 11.3 Neural Networks

  12. Backpropagation and Sequential Neural Networks Reading: ESL 11.4 Fitting Neural Networks. ESL 11.5 Some Issues in Training Neural Networks.

  13. Linear Dimensionality Reduction and SVD Reading: ESL Section 14.15.1 (p.534-536).

  14. Principal Components Analysis Reading: Bishop 12.1 Principal Component Analysis (p.559-569).

  15. Sparse Coding, NMF, ICA and Kernel PCA Reading: ESL Section 14.6 (p.553-557). ESL Section 14.7 (p.557-570).

  16. Clustering I Reading: ESL 14.3.4 - 14.3.11 (k=means). ESL 8.5 (EM).

  17. Clustering II

Exam exception policy: If you have any special needs/circumstances pertaining to an exam, you must talk to the instructor at least 2 weeeks before the exam.

Late homework policy: If you cannot turn in a homework on time, you will need to discuss with the instructor at least one day in advance.

Regrade policy: Any requests for regrading must be submitted within a week of receiving the grade and preferably discussed during office hours. Each TA will be responsible for a different part of the homework, as indicated when the assignment is issued, so please direct questions appropriately. Only contact the instructors after discussing the issue with the TAs.

Copyright/distribution notice: Many of the materials created for this course are the intellectual property of the course instructors and of the professors whose courses served as a basis for some of the lectures. This includes, but is not limited to, the syllabus, lectures and course notes. Except to the extent not protected by copyright law, any use, distribution or sale of such materials requires the permission of the instructor. Please be aware that it is a violation of university policy to reproduce, for distribution or sale, class lectures or class notes, unless copyright has been explicitly waived by the faculty member.