CS 685, Fall 2020, UMass Amherst


Make sure to reload this page to ensure you're seeing the latest version. Readings should be done before watching the corresponding lecture videos.

Week 1 (8/24-28): introduction, language models, representation learning

Week 2 (8/31-9/4): neural LMs, RNNs, backpropagation

Week 3 (9/7-11): Attention mechanisms

Week 4 (9/14-18): Transformers, transfer learning

Week 5: BERT and how to use it for downstream tasks

Week 6: further improving transfer learning in NLP

Week 7: improving text generation

Week 8: data augmentation and collection

Week 9: model distillation and retrieval-augmented LMs

Week 10: Transformer implementation, vision + language

Weeks 10-12: topics TBD