Lecture Times: Tuesdays and Thursdays, 4:00pm-5:15pm Eastern
Lecture: 4:00pm-5:15pm Tuesdays and Thursdays in room 140 of the CS Building
The course provides an introduction to machine learning algorithms and applications. Machine learning algorithms answer the question: "How can a computer improve its performance based on data and from its own experience?" The course is roughly divided into thirds: supervised learning (learning from labeled data), reinforcement learning (learning via trial and error), and real-world considerations like ethics, safety, and fairness. Specific topics include linear and non-linear regression, (stochastic) gradient descent, neural networks, backpropagation, classification, Markov decision processes, state-value and action-value functions, temporal difference learning, actor-critic algorithms, the reward prediction error hypothesis for dopamine, connectionism for philosophy of mind, and ethics, safety, and fairness considerations when applying machine learning to real-world problems.
The TAs and graders for this course are Simon Andrews (sbandrews@umass.edu), Hitesh Golchha (hgolchha@umass.edu), and Sai Phani Teja Vallabhaneni (saiphaniteja@umass.edu). All three will be answering questions on Piazza and Simon will hold office hours.
Office hours will be at the following times:
| Day | Time | Person | Location |
|---|---|---|---|
| Monday | 11:00am-noon | Prof. Thomas | Zoom |
| Tuesday | 2:00pm-3:00pm | Simon Andrews | LGRT 225 |
| Wednesday | 1:00pm-2:00pm | Philip Thomas | CS 346 |
Office hours will follow the UMass Academic Calendar [link].
Assignments will be posted here when they have been assigned.
| Lecture | Topic | Reading |
|---|---|---|
| 1 | Introduction | Chapter 1 |
| 2 | Regression, k-Nearest Neighbors, Linear Regression I | Chapter 2 |
| 3 | Linear Regression II | Chapter 3 |
| 4 | Linear Regression III, Gradient Descent | Chapter 4 |
| 5 | Gradient Descent (continued) | Chapter 5 |
| 6 | Convergence of gradient descent | Chapter 6 |
| 7 | Basis functions, feature normalization, perceptrons | Chapter 7 |
| 8 | Perceptrons | Chapter 8 |
| 9 | Artificial Neural Networks | Chapter 9 |
| 10 | Backpropagation | Chapter 10 |
| 11 | Vanishing Gradients | Chapter 11 |
| 12 | Supervised Learning - Other Topics | Chapter 12 |
| Lecture | Topic | Reading |
|---|---|---|
| 13 | Introduction | Chapter 13 |
| 14 | MENACE, Notation, and Problem Formulation | Chapter 14 |
| 15 | Episodes and Policy Representations | Chapter 15 |
| 16 | MENACE-like RL Algorithm | Chapter 16 |
| 17 | Value functions and TD error | Chapter 17 |
| 18 | Actor-Critics, Options, and Off-Policy Evaluation | Chapter 17 |
| 19 | Review | No readings |
| Lecture | Topic | Reading |
|---|---|---|
| Connections to psychology and neuroscience | Sutton and Barto Chapters 14 and 15 [link] A Neural Substrate of Prediction and Reward [link] Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control [link] Gero Miesenboeck TED Talk [link] |
|
| Fairness, Accountability, and Transparency | TBD | |
| Philosophy of Mind | TBD | |
| Ethics | TBD | |
| Ethics and Safety | TBD | |
| Final Exam Review | TBD |