CS 685, Spring 2025, UMass Amherst
Assignments
-
Homework 0 released, due 2/14
- Quiz 1 released, due 2/17
- Quiz 2 released, due 3/3
-
Final project proposals due 3/7, use this Overleaf template
-
Homework 1 released, due 3/14
- Quiz 3 released, due 3/17
-
Homework 2 released, due 4/11
- Quiz 4 released, due 5/5
-
Extra credit talk summaries due by 5/9, use this Overleaf template
- Final project reports due 5/9, use this Overleaf template
-
Extra credit lecture summaries due by 5/12, use this Overleaf template
Schedule
Make sure to reload this page to ensure you're seeing the latest version.
Readings should be done before watching the corresponding lecture videos. See this page or this page for materials (videos / slides / reading) from the Spring 2023/2024 offering.
Week 1 (2/3-5): Introduction, Language Modeling
-
- Course introduction // [video] (last year) // [slides]
- HW 0 released here, due 2/14
- Final projects:
- Organize into groups of 4 by 2/14. If you do not know people, you can post your interests in Piazza to find partners. Once decided, please fill https://forms.gle/PKvJRxZkUMgFrkVG8 telling us your team members (please sending only one response per group).
- Proposal due 3/7 on Gradescope, use this Overleaf template
- [optional reading] The Bitter Lesson
Week 2 (2/10-12): Neural Language models, Optimization and Backpropagation
Week 3 (2/19): Attention Mechanisms
Week 4 (2/24-26): Transformer
Week 5 (3/3-5): Fine-Tuning and Instruction Tuning
- BERT + Instruction tuning // [video] (last year) // [notes] // [slides]
- [reading] BERT: Pre-training of Deep Bidirectional Transformers... (Devlin et al., 2019)
- [reading] Exploring the Limits of Transfer Learning... (Raffel et al., JMLR 2020, "T5")
- [reading] A Closer Look at the Limitations of Instruction Tuning (Ghosh et al., ICML 2024)
Week 6 (3/10-12): LLM Alignment
Week 7 (3/17-19): Spring Break
Week 8 (3/24-26): Reasoning and Tokenization
Week 9 (3/31-4/2): Decoding and Positional Embedding
Week 10 (4/7-4/9): Midterm Review
Week 11 (4/14-4/18):
Scaling Law, Evaluation, and Midterm
Week 12 (4/23):
Prompt engineering, In-context learning
Week 13 (4/28-4/30):
Agents, Interpretability, Security