Andrew McGregor

Associate Professor

Welcome to the Spring 2021 homepage for CMPSCI 514 - Algorithms for Data Science. See Moodle page for links to lecture recordings, syllabus, homework etc. Slides for future lectures may be motified signifantly depending on our progress. Slides for previous lectures may be updated if, e.g., we spot a typo during the lecture.

Date Topic Reading and Background
2 Feb Course overview. Probability review. Slides
4 Feb Estimating set size by counting duplicates. Concentration Bounds: Markov's inequality. Random hashing for efficient lookup. Slides
9 Feb Finish up hash tables. 2-universal and pairwise independent hashing. Hashing for load balancing. Slides
11 Feb Concentration Bounds Continued: Chebyshev's inequality. The union bound. Exponential tail bounds (Bernstein's inequality). Slides
16 Feb Finish up exponential concentration bounds and the central limit theorem. Bloom filters and their applications. Slides
18 Feb Finish up Bloom filters. Start on streaming algorithms. Min-Hashing for distinct elements. Slides
23 Feb Finish up distinct elements and the median trick. Flajolet-Martin and HyperLogLog. Jaccard similarity estimation with MinHash for audio fingerprinting, document comparision, etc. Start on locality sensitive hashing and nearest neighbor search. Slides
25 Feb Finish up MinHash for Jaccard similarity and locality sensitive hashing. Similarity search. SimHash for Cosine similarity. Slides
2 Mar The frequent elements problem and count-min sketch. Slides
4 Mar Dimensionality reduction, low-distortion embeddings, and the Johnson Lindenstrauss Lemma. Slides
9 Mar Finish up the JL Lemma. Example application to clustering. Connections to high-dimensional geometry. Slides
11 Mar Finish up high-dimensional geometry and connection to the JL Lemma. Slides
16 Mar Intro to principal component analysis, low-rank approximation, data-dependent dimensionality reduction. Slides
18 Mar Projection matrices and best fit subspaces. Slides
23 Mar Optimal low-rank approximation via eigendecomposition. Principal component analysis. Slides
25 Mar Singular value decomposition (SVD) and connections to PCA/low-rank approximation. Slides
30 Mar Applications of low-rank approximation beyond compression. Matrix completion, LSA, and word embeddings. Slides
1 Apr Linear algebraic view of graphs. Spectral graph partitioning. Slides
6 Apr Finish spectral clustering. The stochastic block model. Slides
8 Apr Finish the stochastic block model. Slides
13 Apr Computing the SVD: power method, Krylov methods. Connection to random walks and Markov chains. Slides
15 Apr Start on optimization and gradient descent analysis for convex functions. Slides
22 Apr Finish gradient descent analysis. Constrained optimization and projected gradient descent. Slides
27 Apr Online learning and regret. Online gradient descent. Slides
29 Apr Finish up online gradient descent and stochastic gradient descent analysis. Course conclusion/review. Slides
4 May Extra Class Slides