SPRING 2007: CMPSCI 791BB: REPRESENTATION LEARNING
Instructor: Sridhar Mahadevan
TIME: Friday 1:25-3:55, Room 140
ABSTRACT
The course will be organize into three sections. Part 1 will cover
elementary techniques of representation learning, beginning with
classical linear algebraic methods such as principal components
analysis (PCA). Part 2 will introduce more recent nonlinear
graph-theoretic methods, including global "Fourier" methods such as
Laplacian eigenmaps and "Diffusion Wavelet" methods. Finally, Part 3
will explore advanced topics, including the representation theory of
finite and infinite groups, Lie algebras and Lie groups, and
Riemannian manifolds.
BASIC REPRESENTATION THEORY
Matrix Theory : Eigenspace analysis, orthogonalization
methods, including QR and Gram-Schmidt, low rank approximations,
Krylov subspaces, minimum-norm projections, preconditioners,
perturbation analysis.
Algorithms : Principal components analysis (PCA),
Multi-dimensional scaling (MDS), Random Projections (RP).
Applications : Face recognition (eigenfaces), IR (Latent
semantic indexing), dimensionality reduction.
INTERMEDIATE REPRESENTATION THEORY
Spectral Graph Theory : graph Laplacian, Fiedler
eigenvector, Rayleigh quotient, Cheeger constant, Neumann and
Dirichlet conditions, graph embedding, graph partitioning, expander
graphs, random walks, diffusion processes on graphs
Fourier and wavelet analysis on graphs : local vs. global
basis functions, multi-resolution analysis, scaling functions vs
wavelets, nonlinear approximation.
Algorithms : LLE, ISOMAP, Laplacian and Hessian
Eigenmaps, Diffusion wavelets, Label progagation, Kernel PCA, Nystrom
interpolation, spectral clustering, ICA (independent component analysis).
Applications : 3D object compression in computer
graphics; Markov decision processes: Proto-value functions,
representation policy iteration, compact multi-step models; transfer;
Semi-supervised learning in text and vision; Graph partitioning;
Spectral planning and search.
ADVANCED REPRESENTATION THEORY
Group theory : permutation groups, homomorphisms, graph
groups, automorphisms on graphs, Lie algebras and Lie groups.
Riemannian Manifolds : Smooth manifolds, charts,
coordinate-free analysis.
Algorithms : Fast Fourier Transforms on groups, wavelets
on groups.
Applications : Ranking and selection for information
retrieval;
LOAD : Weekly reading. Ocassional "learning by doing"
problem sets or MATLAB assignments. Research presentations.