SPRING 2007: CMPSCI 791BB: Representation Learning

Principal Instructor: Sridhar Mahadevan

Room 140, Friday 1:25-3:55 (tentative)

PREREQUISITE: CMPSCI 689 (or permission of instructor)

Note: If you took 791BB before, you can still take this class for credit as it covers new topics

Detailed Course Outline


This course will explore new frontiers of research in machine learning that may have a fundamental and far-reaching impact on AI and computer science over the next decade. In particular, we will explore recent methods that dynamically construct feature spaces facilitating the solution of not just a specific problem, but rather of an entire suite of related problems across multiple learning modalities.

Representation learning fundamentally differs from previous formulations of learning, in that it is both modality and task independent. The goal of representation learning is to "understand" the data or state space, or more precisely, estimate the underlying geometry or manifold structure of a data space through the construction of basis functions . These basis functions can then be used in conjunction with standard learning techniques, such as clustering, regression, reinforcement learning, or supervised learning, to solve not just a single problem, but many problems. This course will provide a gentle introduction to both the theory and applications of representation learning.

The course will be organize into three sections. Part 1 will cover elementary techniques of representation learning, beginning with classical linear algebraic methods such as principal components analysis (PCA). Part 2 will introduce more recent nonlinear graph-theoretic methods, including global "Fourier" methods, such as Laplacian eigenmaps, and multi-scale "Wavelet" methods, such as diffusion wavelets. Finally, Part 3 will explore advanced topics, including the representation theory of finite and infinite groups, Lie groups, and Riemannian manifolds. The underlying theory and algorithms will be illustrated using a diverse set of applications, such as compression of 3D objects in computer graphics (see above), ranking (IR), document clustering, analysis of Markov chains and Markov decision processes, sensor networks, computer vision, robotics, networking, and web search.