Course Schedule

Lecture recordings from Echo360 can be accessed here.

Lecture      Day Topic Materials/Reading
Foundations and Random Hashing
1. 2/1 Thu Course overview. Randomized complexity classes and different types of randomized algorithms. Slides. Compressed slides. Reading: Chapters 1-3 of Mitzenmacher Upfal.
2. 2/6 Tue Probability review. Linearity of expectation and variance. Application to polynomial identity testing randomized quicksort analysis. Slides. Compressed slides. Reading: Chapters 1-3 of Mitzenmacher Upfal. Greg Valiant's course notes covering polynomial identity testing (Lec 1) and randomized Quicksort (Lec 2).
3. 2/8 Thu Concentration bounds: Markov's and Chebyshev's inequalities. Union bound. Applications to statistical estimation and coupon collecting. Start on balls-into-bins. Slides. Compressed slides. Reading: Chapters 1-3 of Mitzenmacher Upfal. Greg Valiant's course notes covering concentration bounds (Lec 5).
4. 2/13 Tue Continue on concentration bounds and balls-into-bins. Start on exponential tail bounds (Chernoff and Bernstein inequalities). Slides. Compressed slides. Reading: Chapter 5 of Mitzenmacher Upfal covering balls-into-bins analysis.
5. 2/15 Thu Exponential tail bounds with applications to balls-into-bins and linear probing analysis. Slides. Compressed slides. Reading: Jelani Nelson's course notes, giving a proof of the Chernoff bound (Lec 3) and the linear probing analysis (Lec 4).
6. 2/20 Tue Randomized hash functions and fingerprints. Applications to efficient communication protocols and Rabin-Karp pattern matching Slides. Compressed slides. Reading: Chapter 7 of Motwani Raghavan, with coverage of fingerprinting and its applications.
2/22 Thu No Class, Monday Schedule.
Random Sketching and Randomized Numerical Linear Algebra
7. 2/27 Tue ℓ0 sampling with applications to graph sketching. Slides. Compressed slides. Reading: Many helpful slides/notes by Prof. McGregor on graph sketching algorithms. A good survey on ℓ0 sampling methods.
8. 2/29 Thu Finish ℓ0 sampling. Application to graph streaming. Other linear sketching algorithms -- Count Sketch for frequency estimation. Slides. Compressed slides. Reading: Notes on streaming algorithms by Amit Chakrabarti, including coverage of Count Sketch in Chapter 4 and many related techniques.
9. 3/5 Tue Finish up ℓ2 heavy-hitters via Count Sketch. Start on randomized numerical linear algebra -- approximate matrix multiplication via sampling. Slides. Compressed slides. Reading: Notes on streaming algorithms by Amit Chakrabarti, including coverage of Count Sketch in Chapter 4. Notes on approximate matrix multiplication (Mahoney). General notes on RandNLA, (Drineas and Mahoney) with good background material and coverage of central techniques.
10. 3/7 Thu Finish up approximate matrix multiplication and importance sampling. Application to randomized low-rank approximation. Slides. Compressed slides. Reading: General notes on RandNLA, (Drineas and Mahoney). Some notes on sampling based low-rank approximation (Mahoney).
11. 3/12 Tue Finish up randomized low-rank approximation. Stochastic trace estimation. Slides. Compressed slides. Reading: Notes (Tropp) covering stochastic trace estimation. A blog post  (Nowozin) listing lots of nice applications of randomized trace estimation in machine learning. Another nice blog post  (Epperly) on stochastic trace estimation.
12. 3/14 Thu Finish stochastic trace estimation. Slides. Compressed slides. Reading: Our paper on Hutch++ which gives background on randomized trace estimation and an improved algorithm for PSD matrices.
3/19 Tue No Class, Spring Recess.
3/21 Thu No Class, Spring Recess.
13. 3/26 Tue Midterm Review Slides.
3/28 Thu Midterm Study guide and review questions.
14. 4/2 Tue The Johnson-Lindenstrauss lemma, epsilon-nets, and subspace embedding. Slides. Compressed slides. Reading: Chris Muscos's notes on subspace embedding, on which my proof in class is based.
15. 4/4 Thu Finish up subspace embedding. Proof of the JL Lemma via Hanson-Wright. Sketching for linear regression. Slides. Compressed slides. Reading: Jelani Nelson's course notes, which include the proof of distributional JL via Hanson-Wright, along with discussion of subspace embeddings and various applications.
16. 4/9 Tue Importance sampling and leverage scores. Matrix concentration bounds. Slides. Compressed slides.
17. 4/11 Thu Finish up leverage scores. Connection to effective resistences and spectral graph sparsifiers. Slides. Compressed slides. Reading: Dan Spielman's great book on spectral graph theory, including Chapter 32 on spectral sparsification via sampling, and analysis via matrix Chernoff bounds, which is equivilant to the leverage score analysis shown in class.
4/16 Tue No Class, Prof Traveling.
Markov Chains
18. 4/18 Thu Prof Traveling. Class held over Zoom. Introduction to Markov chains and their analysis. Application to randomized algorithms for 2-SAT and 3-SAT. Slides. Compressed slides.. Lecture Recording. Reading: Chapter 7 of Mitzenmacher Upfal on Markov Chains, including the application to 2-SAT and 3-SAT.
19. 4/23 Tue Gambler's ruin. Markov chain analysis and stationary distributions. The Fundamental Theorem of Markov Chains. Slides. Compressed slides. Reading: Chapter 7 of Mitzenmacher Upfal.
20. 4/25 Thu Analysis of Markov chain mixing time via coupling. Slides. Compressed slides. Reading: Notes by Greg Valiant and Mary Wooters covering mixing times/coupling. Also Chapters 10/11 of Mitzenmacher Upfal.
21. 4/30 Tue Finish up coupling. Start on Markov chain Monte Carlo methods (MCMC). Slides. Compressed slides. Reading: Notes by Greg Valiant and Mary Wooters covering the Metropolis-Hastings algorithm. Chapter 10/11 of Mitzenmacher Upfal covering MCMC and approximate counting.
22. 5/2 Thu Finish up Markov Chains. Metropolis Hastings, sampling to counting reductions. Slides. Compressed slides. Reading: Notes by Greg Valiant and Mary Wooters covering the Metropolis-Hastings algorithm. Chapter 10/11 of Mitzenmacher Upfal covering MCMC and approximate counting.
Other Topics
23. 5/7 Tue Approximation algorithms via convex relaxation + randomized rounding. Applications to vertex cover, set cover, and max-cut Slides. Compressed slides. Reading: Lecture notes by Chekuri covering the relaxations for vertex cover and set cover presented in class. Book on approximation algorithms by Williamson and Shmoys, with lots of coverages of relaxation and rounding.
24. 5/9 Thu Probabilistic method and the Lovasz Local Lemma. Slides. Compressed slides. Reading: Chapter 6 of Mitzenmacher Upfal. Greg Valiant's lecture notes on probabilistic method and algorithmic Lovasz local lemma.
5/14 Tue Final Exam. 10:30am - 12:30pm. Location TBD Study guide and review questions.