COMPSCI 589: Machine Learning, Fall 2018

Course Description: This course will introduce core machine learning models and algorithms for classification, regression, clustering, and dimensionality reduction. On the theory side, the course will focus on understanding models and the relationships between them. On the applied side, the course will focus on effectively using machine learning methods to solve real-world problems with an emphasis on model selection, regularization, design of experiments, and presentation and interpretation of results.

Override questions: If you'd like to take this course but cannot register, please submit an override request through the online system. Above all, please describe your background in linear algebra, probability theory, and multivariate calculus. Please list any courses you've taken either in those topics, or using those topics or any other relevant training or experience you might have. Typically we have strong interest in this course, so we unfortunately can't guarantee everyone will be able to take it.

Textbooks: The course readings will primarily be based on two open textbooks:

• The Elements of Statistical Learning by Hastie, Tibshirani and Friedman
• An Introduction to Statistical Learning by James, Witten, Hastie and Tibshirani

Required Background: This course requires appropriate mathematical background in probability and statistics, calculus, and linear algebra. The official prerequisites for undergrads are CS 383 and MATH 235 (CS 240 provides sufficient background in probability and MATH 131/132 provide sufficient background in calculus).

Graduate students can check the descriptions for these courses to verify that they have sufficient mathematical background for 589. The course will also use Python as a programming language including the numpy, scipy, and scikit-learn. Some familiarity with Python will be helpful, but senior CS students should be able to learn Python during the course if needed. Graduate students from outside computer science with sufficient background are also welcome to take the course.

The following references can provide a useful review:

• Probability Theory
• Linear Algebra and Matrix Calculus
• Matrix Cookbook
• Matrix Identities

More specific topics to be comfortable with, for this course:

• Linear algebra: vectors and matrices, types of matrices (square, diagonal, identity), linear operations (scalar products, matrix multiplication, etc.), other transforms and computations (transpose, inverse, eigenvalues)
• Probability: laws of probability, Bayes Rule, CDFs, PDF vs PMF, etc.
• Calculus: derivatives, integrals, analytic minimization, etc.

What is the difference between CS 589 and CS 689?: 589 has been designed to focus on understanding and applying core machine learning models and algorithms, while 689 focuses on the mathematical foundations of machine learning. Both courses require a background in multivariate calculus, linear algebra, and probability, but 689 is more theoretically focused and will use more of this background material than 589. In particular, 589 will not focus on deriving learning or optimization algorithms.

Should I take CS 589 or CS 689?: 589 is appropriate as an introductory machine learning course for senior undergraduate students, masters students, and MS/PhD students interested in applying machine learning in their research. Note that 589 can count for credit for MS/PhD students, but it does not satisfy an AI core requirement. Graduate students who intend to pursue research in machine learning or who need a course to satisfy the AI core requirement should take 689.

Course requirements:

• 50% Homeworks (5 total). These include both written and math questions, as well as programming and Kaggle submission problems.
• 20% In-class quizzes (~5 total, ~10 minutes each). First quiz is Monday, 9/10, on background topics.
• 30% Final exam

There will be 3 late days (24 hour periods after the due date).

Major topics:

• Regression models
• Classification models
• (Running themes: generalization, parametric vs. non-parametric, etc.)
• Kernel methods
• Bayesian methods
• Unsuprvised learning

Collaboration policy

• All of the content you submit, both code and text, needs to be produced independently. Your work must be in your own words and based on your understanding of the solution. ("Independently" applies to homework; for group projects, work must be produced only by members of the group.)
• Do not share code or written materials. Do not look at others' code. You may discuss problems and the project with others, and we encourage it, to help understand the material.
• On your homework, list the names of everyone you collaborated or had discussions with.
• If you find, use, or build off of published material, for example on the web or from a textbook, you must cite the source. Always explain the answer in your own words.

Some examples of the policy:

• Acceptable: Alice and Bob discuss alternatives for storing large, sparse vectors of feature counts, as required by a problem set.
• Unacceptable: Alice and Bob sit down together and write code for storing feature counts.
• Acceptable: Bob is confused about how to implement the Viterbi algorithm, and asks Alice for a conceptual description of her strategy.
• Unacceptable: Alice and Bob divide the assignment into parts, and each write the code for their part, and then share their solutions with each other to complete the assignment.
• Acceptable: Alice asks Bob if he encountered a failure condition at a "sanity check" in a coding assignment, and Bob explains at a conceptual level how he overcame that failure condition.
• Unacceptable: Alice or Bob obtain a solution to a previous year's assignment or to a related assignment in another class, and use it as the starting point for their own solution.

We follow the university’s Academic Honesty Policy and Procedures.

If you have questions about a particular situation, please ask.