Philip S. Thomas

COMPSCI 389: Introduction to Machine Learning

Spring 2022, University of Massachusetts

Lecture Times: Tuesdays and Thursdays, 4:00pm-5:15pm Eastern

Archived course page. Dates, office hours, and course logistics are retained for historical reference.

Course Information

Download syllabus (.pdf)

Lecture: 4:00pm-5:15pm Tuesdays and Thursdays in room 140 of the CS Building

Description

The course provides an introduction to machine learning algorithms and applications. Machine learning algorithms answer the question: "How can a computer improve its performance based on data and from its own experience?" The course is roughly divided into thirds: supervised learning (learning from labeled data), reinforcement learning (learning via trial and error), and real-world considerations like ethics, safety, and fairness. Specific topics include linear and non-linear regression, (stochastic) gradient descent, neural networks, backpropagation, classification, Markov decision processes, state-value and action-value functions, temporal difference learning, actor-critic algorithms, the reward prediction error hypothesis for dopamine, connectionism for philosophy of mind, and ethics, safety, and fairness considerations when applying machine learning to real-world problems.

Download Course Notes .pdf

TAs and Office Hours

The TAs and graders for this course are Simon Andrews (sbandrews@umass.edu), Hitesh Golchha (hgolchha@umass.edu), and Sai Phani Teja Vallabhaneni (saiphaniteja@umass.edu). All three will be answering questions on Piazza and Simon will hold office hours.

Office hours will be at the following times:

Day Time Person Location
Monday 11:00am-noon Prof. Thomas Zoom
Tuesday 2:00pm-3:00pm Simon Andrews LGRT 225
Wednesday 1:00pm-2:00pm Philip Thomas CS 346

Office hours will follow the UMass Academic Calendar [link].

Assignments

Assignments will be posted here when they have been assigned.

  1. Homework 1 has been assigned on 1 February 2022 and is due at 4pm on 10 February 2022. [link (.zip)]
  2. Homework 2 has been assigned on 15 February 2022 and is due at 11:59pm on 27 February 2022. [link (.zip)]
  3. Homework 3 has been assigned on 3 March 2022 at 4:00pm and is due at 11:59pm on 10 March 2022. [link (.zip)]
  4. Homework 4 has been assigned on 24 March 2022 and is due at 11:59pm on 5 April 2022. [link (.zip)]
  5. Homework 5 has been assigned on 12 April 2022 and is due at 11:59pm on 22 April 2022. [link (.zip)]
  6. Homework 6 has been assigned on 22 April 2022 and is due at 11:59pm on 3 May 2022. [link (.zip)]

Schedule

Part I: Supervised Learning

Lecture Topic Reading
1 Introduction Chapter 1
2 Regression, k-Nearest Neighbors, Linear Regression I Chapter 2
3 Linear Regression II Chapter 3
4 Linear Regression III, Gradient Descent Chapter 4
5 Gradient Descent (continued) Chapter 5
6 Convergence of gradient descent Chapter 6
7 Basis functions, feature normalization, perceptrons Chapter 7
8 Perceptrons Chapter 8
9 Artificial Neural Networks Chapter 9
10 Backpropagation Chapter 10
11 Vanishing Gradients Chapter 11
12 Supervised Learning - Other Topics Chapter 12

Part II: Reinforcement Learning

Lecture Topic Reading
13 Introduction Chapter 13
14 MENACE, Notation, and Problem Formulation Chapter 14
15 Episodes and Policy Representations Chapter 15
16 MENACE-like RL Algorithm Chapter 16
17 Value functions and TD error Chapter 17
18 Actor-Critics, Options, and Off-Policy Evaluation Chapter 17
19 Review No readings

Part III: Ethics, Safety, Fairness, and Connections to other Areas

Lecture Topic Reading
Connections to psychology and neuroscience Sutton and Barto Chapters 14 and 15 [link]
A Neural Substrate of Prediction and Reward [link]
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control [link]
Gero Miesenboeck TED Talk [link]
Fairness, Accountability, and Transparency TBD
Philosophy of Mind TBD
Ethics TBD
Ethics and Safety TBD
Final Exam Review TBD