Interactive Machine Learning: Algorithms and Theory

CMPSCI 691E, Fall 2016

Akshay Krishnamurthy

Overview: Interactive machine learning involves an algorithm or an agent making decisions about data collection, contrasting starkly with traditional learning paradigms. Interactive data collection often enables learning with significantly less data, and it is critical in a number of applications including personalized recommendation, medical diagnosis, and dialogue systems. This seminar will focus on the design and analysis of interactive learning algorithms for settings including active learning, bandits, reinforcement learning, and adaptive sensing. We will cover foundational and contemporary papers, with an emphasis on algorithmic design principles as well as understanding and proving performance guarantees.

Requirements (3 units): Students enrolled in the 3 unit version of the course are required to present one paper in lecture and complete a small research project in interactive learning, in addition to reading the papers, attending class, and engaging in discussion. Presentation of a paper should include proofs of the main results. Research projects can comprise original research or a literature survey on any topic related to interactive learning. Project guidelines are available here.

Requirements (1 unit): Students enrolled in the 1 unit version of the course are expected to read the assigned papers, attend class, and engage in discussions.

Tentative Syllabus

Date	Lecture Topics	Readings	Presenter
9/6	Stochastic Bandits	Chapter 2 of Bubeck and Cesa-Bianchi	Akshay
9/13	Online Learning and Adversarial Bandits	Chapter 3 of Bubeck and Cesa-Bianchi	Soumyabrata Pal
9/20	Special Bandit Topics	Thompson Sampling Gaussian Process Bandits	Ryan McKenna
9/27	Parametric Bandits	LinUCB	Akshay
10/4	Contextual Bandits	EXP4 Epoch-Greedy	Raj Kumar Maity
10/11	NO CLASS
10/18	Model-Based PAC Reinforcement Learning	E-cubed	Tengyang Xie
10/25	Reinforcement Learning Regret Bounds	UCRL2	Akshay
11/1	Model-Free PAC Reinforcement Learning	Delayed-Q Learning	Akshay
11/8	Selective Sampling/Parametric Active Learning	Better algorithms for selective sampling	Haw-Shiuan Chang
11/15	Active Learning Survey	Two faces of active learning	Craig Greenberg
11/22	NO CLASS
11/29	Agnostic Active Learning	CAL Daniel Hsu's note	Yuqing Xing
12/6	NO CLASS
12/13	Agnostic Active Learning	IWAL	Akshay

Additional Reading: Some other papers that may be of interest are listed here.