Welcome to the Fall 2024 homepage for COMPSCI 688: Probabilistic Graphical Models. Below find basic information, coursework and schedule, and course policies.
| Course name | COMPSCI 688 Probabilistic Graphical Models |
| Instructor | Dan Sheldon, (email: sheldon at cs) |
| Delivery Mode | In person |
| Lecture | Monday, Wednesday 4:00–5:15pm, Engineering Lab Room 304 |
| Credits | 3 |
| Canvas | https://umamherst.instructure.com/courses/24080 |
| Gradescope | https://www.gradescope.com/courses/852044 |
| Piazza | https://piazza.com/umass/fall2024/compsci688 |
| Echo 360 | https://umamherst.instructure.com/courses/24080/external_tools/147 |
| TAs | Iman Deznabi (ideznabi at umass.edu) Shakir Sahibul (ssahibul at umass.edu) |
| Office Hours | See list on Piazza (will be updated throughout semester) |
There will be five homework assignments during the semester. Assignment packages will be posted on Canvas roughly 2 weeks before the due date and submitted via Gradescope.
The final exam is on Monday, December 16, 2024 from 3:30–5:30pm in Engineering Lab Room 304.
Here is an approximate schedule and list of topics for the course. This is subject to change and will be updated as we go. Slides will be added after class and links will be broken until they are added.
| Week | Lecture | Date | Topic | Materials |
|---|---|---|---|---|
| 1 | 1 | 9/4 | Course overview | Slides: nup, annotated |
| 2 | 2 | 9/9 | Background + Bayesian networks | Slides: nup, annotated |
| 3 | 9/11 | Bayesian networks | Slides: nup, annotated | |
| 3 | 4 | 9/16 | D-Separation | Slides: nup, annotated |
| 5 | 9/18 | Learning in Bayes nets | Slides: nup, annotated, whiteboard | |
| 4 | 6 | 9/23 | Undirected models | Slides: nup, annotated |
| 7 | 9/25 | Undirected models | Slides: nup, annotated | |
| 5 | 8 | 9/30 | Inference | Slides: nup, annotated |
| 9 | 10/2 | Message passing | Slides: nup, annotated | |
| 6 | 10/7 | NO CLASS - INSTRUCTOR TRAVEL | ||
| 10 | 10/9 | Learning in undirected models | Slides: nup, annotated | |
| 7 | 10/14 | NO CLASS - INDIGENOUS PEOPLE’S DAY | ||
| 11 | 10/15 | Exponential families (Monday schedule) | Slides: nup, annotated | |
| 12 | 10/16 | Exponential families | Slides: nup, annotated | |
| 8 | 13 | 10/21 | Markov chain Monte Carlo | Slides: nup, annotated |
| 14 | 10/23 | MCMC foundations | Slides: nup, annotated | |
| 9 | 15 | 10/28 | Gibbs and Metropolis-Hastings | Slides: nup, annotated |
| 16 | 10/30 | MCMC additional topics | Slides: nup, annotated | |
| 10 | 17 | 11/4 | Conjugate Bayesian inference | Slides: nup, annotated |
| 18 | 11/6 | Hamiltonian Monte Carlo | Slides: nup, annotated | |
| 11 | 11/11 | NO CLASS - VETERAN’S DAY | ||
| 19 | 11/13 | Variational inference | Slides: nup, annotated | |
| 12 | 20 | 11/18 | Black-box VI | Slides: nup, annotated |
| 21 | 11/20 | Variational autoencoders | Slides: nup, annotated | |
| 13 | 21.5 | 11/25 | Variational autoencoders continued | (no new slides) |
| 11/27 | NO CLASS - THANKSGIVING | |||
| 14 | 22 | 12/2 | Gaussian Processes | Slides: nup, annotated demo |
| 23 | 12/4 | Normalizing flows | Slides: nup, annotated | |
| 15 | 24 | 12/9 | Advanced topics / final review | Slides: nup, annotated |
For futher information, including readings and other source materials corresponding to the topics above, please see this list of topics.
Probabilistic graphical models are an intuitive visual language for describing the structure of joint probability distributions using graphs. They enable the compact representation and manipulation of exponentially large probability distributions, which allows them to efficiently manage the uncertainty and partial observability that commonly occur in real-world problems. As a result, graphical models have become invaluable tools in a wide range of areas from computer vision and sensor networks to natural language processing and computational biology. The aim of this course is to develop the knowledge and skills necessary to effectively design, implement and apply these models to solve real problems.
The course will cover (a) Bayesian and Markov networks and their dynamic and relational extensions; (b) exact and approximate inference methods; (c) estimation of both the parameters and structure of graphical models.
Students entering the class should have good programming skills and knowledge of algorithms. Python is strongly recommended. Undergraduate-level knowledge of probability, linear algebra, and calculus will be assumed. A prior course in machine learning is extremely helpful.
Here are some review resources on probability, linear algebra, and Python / SciPy:
There is no required textbook. These two books provide useful supplemental reading:
An additional resource is
I will not assign specific readings for individual lectures. However, there is a partial list of topics that have been covered in previous years with associated readings and source material. Please refer to this list to find reference material, and ask a course staff member if you would like additional pointers.
Official announcements will be made via Piazza. Piazza will also host discussion forums for the class. To contact the course staff, please use a Piazza private post (not email). Here is the link to the CS 688 Piazza page: https://piazza.com/umass/spring2024/compsci688. We will attempt to respond to communications with 24 hours on weekdays, 48 hours on weekends.
The coursework will consist of homework assignments, quizzes, and a final exam. The assignments will consist of mathematical derivations, programming and experimentation, and some written questions. Assignment solutions will be written as short reports. Quizzes will be administered online using Gradescope. Assignments will be submitted online via Gradescope. The grading percentage breakdown is:
The grading scale will be determined at the end of the semester. An approximate scale is A: 100% to 92%, A– <92% to 87%, B+: <87% to 80%, B: <80% to 75%, B–: <75% to 70%, C+: <70% to 65%, C: <65% to 60%, F: <60%.
Homework assignments will generally consist of written derivations, implementation of machine learning algorithms in a language of your choice (Python is strongly recommended), evaluation of algorithms, and writing of reports. Both the code and report must be submitted by the due date for a submission to be considered on time. Reports must be typed. Assignments will be submitted via Gradescope.
To allow some flexibility to complete assignments given other constraints, you have a total of five free late days. You will be charged one late day for handing in an assignment within 24 hours after it is due, two late days for handing in an assignment within 48 hours after it is due, etc. Your assignment is considered late if either the written or code portions are submitted late. The late homework clock stops when both the written and code portions are submitted. After you have used up your late days, late homework will not count for credit except in special circumstances (i.e., illness documented by a doctor’s note).
Quizzes will be assigned roughly weekly on Gradescope. They will typically be posted at the end of a week and due the following Friday at 11:59pm. A quiz will generally involve solving 1–3 short problems. There is a one hour time limit. Quiz problems are not highly involved, but go beyond basic definitions and short answer questions. The intention is that students prepare by reviewing lecture slides and reading any associated materials before starting the quiz. With this preparation, the expected time to complete a quiz is no more than 30 minutes. There may not be sufficient time to both review material and produce answers within the time limits. The lowest quiz score will be dropped.
Homework assignments are considered individual work. You may discuss the problems with other students; however, to avoid issues with the course’s academic honesty policy, you should not take any materials out of such discussions (writing, whiteboard photos, etc.). You should never share your completed or in-progress code or write-up with another student in any form, or request to see another student’s code or write-up. Your derivations, code, and write-up must be your own work. Quizzes are strictly individual work. No collaboration of any kind is permitted on quizzes.
You are required to list the names of anyone you discuss problems with on the first page of your solution report. Copying any solution materials (derivations, code, method descriptions) from external sources (books, web pages, etc.) or from other students is considered cheating. Sharing your code or solutions with other students is also considered cheating. Collaboration indistinguishable from copying will be treated as copying. All instances of suspected cheating will be dealt with through official UMass Amherst Academic Honesty Procedures. Students are expected to be familiar with the relevant policies and procedures: https://www.umass.edu/studentsuccess/academic-integrity
Since the integrity of the academic enterprise of any institution of higher education requires honesty in scholarship and research, academic honesty is required of all students at the University of Massachusetts Amherst. Academic dishonesty is prohibited in all programs of the University. Academic dishonesty includes but is not limited to: cheating, fabrication, plagiarism, and facilitating dishonesty. Appropriate sanctions may be imposed on any student who has committed an act of academic dishonesty. Instructors should take reasonable steps to address academic misconduct. Any person who has reason to believe that a student has committed academic dishonesty should bring such information to the attention of the appropriate course instructor as soon as possible. Instances of academic dishonesty not related to a specific course should be brought to the attention of the appropriate department Head or Chair. Since students are expected to be familiar with this policy and the commonly accepted standards of academic integrity, ignorance of such standards is not normally sufficient evidence of lack of intent. See https://www.umass.edu/studentsuccess/academic-integrity
If you believe you’ve found a grading error for an assignment or quiz, please submit a regrade request. Regrade requests must be submitted no later than one week after the assignment is returned. Regrading may result in your original grade increasing or decreasing.
Students are expected to attend class and participate in discussions and are responsible for all material presented and all announcements made in class. There is no grade penalty for missed classes. Video lectures will be available on Echo360 for all students 3–4 days after the lecture date; if a student is unable to attend class due to illness or a conflict, they may request access sooner via the CS688 Echo360 request form.
The instructor and the University share intellectual property rights for all course materials including lecture slides, lecture audio/video recordings, demo code, assignment handouts, and exam materials. Students are allowed to keep copies of this material for personal use, but are prohibited from distributing it to other individuals and/or posting it in part or in whole on publicly accessible sites including on slide share sites and sites such as Chegg. Students are not permitted to make their own lecture recordings (audio or video). Official recordings will be made available to all students after a 3–4 day delay.
The University of Massachusetts Amherst is committed to providing an equal educational opportunity for all students. If you have a documented physical, psychological, or learning disability on file with Disability Services (DS), you may be eligible for reasonable academic accommodations to help you succeed in this course. If you have a documented disability that requires an accommodation, please notify me within the first two weeks of the semester so that we may make appropriate arrangements. For further information, please visit https://www.umass.edu/disability/.
In accordance with Title IX of the Education Amendments of 1972 that prohibits gender-based discrimination in educational settings that receive federal funds, the University of Massachusetts Amherst is committed to providing a safe learning environment for all students, free from all forms of discrimination, including sexual assault, sexual harassment, domestic violence, dating violence, stalking, and retaliation. This includes interactions in person or online through digital platforms and social media. Title IX also protects against discrimination on the basis of pregnancy, childbirth, false pregnancy, miscarriage, abortion, or related conditions, including recovery. There are resources here on campus to support you. A summary of the available Title IX resources (confidential and non-confidential) can be found at the following link: https://www.umass.edu/titleix/resources. You do not need to make a formal report to access them. If you need immediate support, you are not alone. Free and confidential support is available 24 hours a day / 7 days a week / 365 davs a vear at the SASA Hotline 413-545-0800.
Please read the CICS inclusivity statement, copied here:
At the Manning College of Information and Computer Sciences, we believe that you belong in computing. We welcome and value all individuals, regardless of previous computer science experience, age, citizenship, disability, sex, gender identity, military experience, political views, race, religion, or sexual orientation, while maintaining an environment that celebrates, welcomes, and honors those differences.
We’re committed to supporting all our students through their journeys in computer and information sciences–especially students from identities and backgrounds that are still underrepresented in our field. Diverse perspectives on the challenges our society faces animate our vision of Computing for the Common Good. Your insight, talents, and skills are needed to protect and improve an ecosystem that relies on the combined efforts of the greatest technical minds, and we believe your place is here.