SPRING 2007: CMPSCI 791BB: REPRESENTATION LEARNING

Instructor: Sridhar Mahadevan

TIME: Friday 1:25-3:55, Room 140

ABSTRACT

The course will be organize into three sections. Part 1 will cover elementary techniques of representation learning, beginning with classical linear algebraic methods such as principal components analysis (PCA). Part 2 will introduce more recent nonlinear graph-theoretic methods, including global "Fourier" methods such as Laplacian eigenmaps and "Diffusion Wavelet" methods. Finally, Part 3 will explore advanced topics, including the representation theory of finite and infinite groups, Lie algebras and Lie groups, and Riemannian manifolds.

BASIC REPRESENTATION THEORY

  • Matrix Theory : Eigenspace analysis, orthogonalization methods, including QR and Gram-Schmidt, low rank approximations, Krylov subspaces, minimum-norm projections, preconditioners, perturbation analysis.
  • Algorithms : Principal components analysis (PCA), Multi-dimensional scaling (MDS), Random Projections (RP).
  • Applications : Face recognition (eigenfaces), IR (Latent semantic indexing), dimensionality reduction.

    INTERMEDIATE REPRESENTATION THEORY

  • Spectral Graph Theory : graph Laplacian, Fiedler eigenvector, Rayleigh quotient, Cheeger constant, Neumann and Dirichlet conditions, graph embedding, graph partitioning, expander graphs, random walks, diffusion processes on graphs
  • Fourier and wavelet analysis on graphs : local vs. global basis functions, multi-resolution analysis, scaling functions vs wavelets, nonlinear approximation.
  • Algorithms : LLE, ISOMAP, Laplacian and Hessian Eigenmaps, Diffusion wavelets, Label progagation, Kernel PCA, Nystrom interpolation, spectral clustering, ICA (independent component analysis).
  • Applications : 3D object compression in computer graphics; Markov decision processes: Proto-value functions, representation policy iteration, compact multi-step models; transfer; Semi-supervised learning in text and vision; Graph partitioning; Spectral planning and search.

    ADVANCED REPRESENTATION THEORY

  • Group theory : permutation groups, homomorphisms, graph groups, automorphisms on graphs, Lie algebras and Lie groups.
  • Riemannian Manifolds : Smooth manifolds, charts, coordinate-free analysis.
  • Algorithms : Fast Fourier Transforms on groups, wavelets on groups.
  • Applications : Ranking and selection for information retrieval;

  • LOAD : Weekly reading. Ocassional "learning by doing" problem sets or MATLAB assignments. Research presentations.