Graphical Models, Spring, 2020


Course Number: COMPSCI 688
Time: MW / 4:00-5:15
Room: Goessmann Laboratory Room 20
Instructor: Justin Domke

Course Description: Probabilistic graphical models are an intuitive visual language for describing the structure of joint probability distributions using graphs. They enable the compact representation and manipulation of exponentially large probability distributions, which allows them to efficiently manage the uncertainty and partial observability that commonly occur in real-world problems. As a result, graphical models have become invaluable tools in a wide range of areas from computer vision and sensor networks to natural language processing and computational biology. The aim of this course is to develop the knowledge and skills necessary to effectively design, implement and apply these models to solve real problems. The course will cover (a) Bayesian and Markov networks and their dynamic and relational extensions; (b) exact and approximate inference methods; (c) estimation of both the parameters and structure of graphical models. Although the course is listed as a seminar, it will be taught as a regular lecture course with programming assignments and exams. Students entering the class should have good programming skills and knowledge of algorithms. Undergraduate-level knowledge of probability and statistics is recommended.

Course Update: (Nov 7) The course is nearly full. If we make any changes regarding the class size, they will be announced here. (In particular, checking this page will be a faster way to find out than emailing the instructor!)

Course Textbook: (none)
We will distribute lecture notes, as well as some readings from papers and online resources.

Prerequisites: I will assume a strong working knowledge of Linear Algebra, Probability theory, programming ability in some language (e.g. Python) and that you have some familiarity with basic machine learning methods.

Topics (approximate)

  • Directed Models
  • Bayes Ball
  • D-Separation
  • Maximum Likelihood in Discrete Directed Models
  • Undirected Models and Hammersley-Clifford Theorem
  • Exact inference via Message-Passing
  • Exponential families and maximum likelihood
  • The Chow-Liu Algorithm
  • Markov chain Monte Carlo
  • Bayesian Inference
  • Variational Inference
  • Gaussian Processes

Should you take this class?: If you are debating if you want to take this class, you can watch this tutorial, which will give you an idea of the instructor's teaching style, and the type of content we will cover. After watching that video, if you think "I would like to learn these things with much more mathematical detail, as well as go into much more advanced topics along the same lines!" Then you might like this course.

Do you want an override?: Please use the online system to request an override to join the course, or to join the waitlist. Please do not directly email the instructor-- this will not help you jump the waitlist (this would be totally unfair to people who simply requested an override in the standard way).

Course Policies

  • Pass Fail / Audit. This class has extreme demand for seats. Thus, I will not approve any students to take the class pass/fail or as audit. Depending on what degree program you are in, you still may be able to register for the class in this way, and I will of course respect the university policy in that regard. However, I will not sign any forms approving a student to take the course pass/fail. Many students also ask to switch to pass/fail several weeks into the semester. I will not approve any such requests. If you want to be able to switch to pass/fail, you should not register for this class.
  • Quiz Solutions. For logistical reasons, we are not able to provide solutions for quizzes. However, you can (and are encouraged to!) go over your quiz answer with a TA in office hours.
  • Quiz Grading. The lowest quiz score will be dropped before computing your average quiz score.
  • Homework Submission: Homework assignments will generally consist of developing machine learning systems in Python, evaluating the systems, and producing written reports. Both the code and report must be submitted by the due date for a submission to be considered on time.
  • Late Homework: To allow some flexibility to complete assignments given other constraints, you have a total of five free late days. You will be charged one late day for handing in an assignment within 24 hours after it is due, two late days for handing in an assignment within 48 hours after it is due, etc. Your assignment is considered late if either the written or code portions are submitted late. The late homework clock stops when both the written and code portions are submitted. After you have used up your late days, late homework will not count for credit except in special circumstances (ie: illness documented by a doctors note). If you do not hand in an assignment at all, this will count as using all five late days.
  • Homework Collaboration: You are encouraged to verbally discuss assignments and course material with other students in person or on the course forums. However, you must show that you fully understand the solution to any homework problem arising from such collaboration by writing your own code, running your own experiments, and producing your own write-up for the problem. You should not at any time look at the solutions of another student. Showing your solutions to another student is also considered cheating.
  • Academic Honesty Policy: You are required to list the names of anyone you discuss problems with on the first page of your solutions. This includes teaching assistants or instructors. Copying any solution materials from external sources (books, web pages, etc.) or other students is considered cheating. To emphasize: no detectable copying is acceptable, even, e.g., copying a single sentence from an outside source. Sharing your code or solutions with other students is also considered cheating. Any detected cheating will result in a grade of -100% on the assignment for all students involved (negative credit), and potentially a grade of F in the course.
  • Re-grading Policy: Errors in grading of assignments and exams can occur despite the best efforts of the course staff. If you believe you've found a grading error, complete the online re- grade request form. Re-grade requests must be submitted no later than one week after the assignment is returned. Note that re-grading may result in your original grade increasing or decreasing as appropriate.