Schedule

Make sure to reload this page to ensure you’re seeing the latest version. Readings should be done before the indicated class. Texts include:

• JM = Jurafsky and Martin, Speech and Language Processing, 3rd edition draft chapters
• Eis. = Eisenstein draft text
• Gol. = Yoav Goldberg's neural networks for NLP primer

Tue 9/3 - Introduction & Python demo // [slides]

• HW 0 released, due next Thursday on Gradescope // LaTeX (in Overleaf)

Thu 9/5 - Text classification I (naive Bayes) // [slides]

• Reading: JM 4.1-4.6

Tue 9/10 - Language modeling I: n-gram models // [slides]

• Reading: JM 3.1-3.5

Thu 9/12 - Language modeling II: log-linear LMs & final projects // [slides]

• Reading: Graham Neubig's notes
• HW 0 due by end of the day!

Tue 9/17 - Learning for log-linear LMs // [slides]

• Reading: same as last class
• HW 1 released, due Sep 30 on Gradescope // Colab notebook here
• Project proposal template released, due Oct 4 on Gradescope // Overleaf here

Thu 9/19 - Neural language models I: word embeddings // [slides]

• Reading: JM 7
• Optional reading: A Neural Probabilistic Language Model, Bengio et al., JMLR 2003
• Optional reading: Distributed Representations of Words and Phrases and their Compositionality, Mikolov et al., NeuRIPS 2013

Tue 9/24 - Neural composition functions + machine translation // [slides]

• Reading: JM 9
• Optional reading: Convolutional Neural Networks for Sentence Classification, Yoon Kim, EMNLP 2014.
• Optional reading: Sequence to Sequence Learning with Neural Networks, Sutskever et al., NeuRIPS 2014

Thu 9/26 - MT (cont.) // [slides]

• Reading: Eis. 18.3, 18.4
• Optional reading: Neural Machine Translation by Jointly Learning to Align and Translate, Bahdanau et al., ICLR 2015
• Optional reading: Attention Is All You Need, Vaswani et al., NeuRIPS 2017

Tue 10/1 - Transformers + PyTorch demo // [slides]

• Same reading as last class
• HW 2 released, due Oct 18 on Gradescope // Colab notebook here

Thu 10/3 - Transformers (cont.) // [slides]

• Reading 1: Deep contextualized word representations, Peters et al., NAACL 2018
• Reading 2: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al., NAACL 2019

Tue 10/7 - Transfer learning with neural language models // [slides]

• Same reading as last class

Thu 10/9 - Using BERT in downstream tasks // [slides]

• Optional reading 1: XLNet: Generalized Autoregressive Pretraining for Language Understanding, Dai et al., 2019
• Optional reading 2: RoBERTa: A Robustly Optimized BERT Pretraining Approach, Liu et al., 2019

Tue 10/15 - No class (Monday class schedule)

Thu 10/17 - Sequence labeling with HMMs // [slides]

• Reading: J+M 8.1-8.4

Tue 10/22 - Viterbi algorithm // [slides]

• Reading: same as before

Thu 10/24 - Ethics in NLP // [slides]

• Reading: The Social Impact of Natural Language Processing, Hovy & Spruit, 2016.
• Optional: Check out Yulia Tsvetkov's class on computational ethics for NLP

Tue 10/29 - Midterm review // [slides]

Thu 10/31 - Midterm!

Tue 11/5 - Guest lecture on unsupervised machine translation by Simeng Sun (Mohit out for EMNLP this week) // [slides]

• Reading: How we trained a translation model from West African Pidgin to English without a single parallel sentence
• Optional reading 1: Word Translation Without Parallel Data, Conneau et al., ICLR 2018
• Optional reading 2: Unsupervised Machine Translation Using Monolingual Corpora Only, Lample et al., ICLR 2018
• Optional reading 3: Phrase-Based & Neural Unsupervised Machine Translation, Lample et al., EMNLP 2018

Thu 11/7 - Guest lecture on implementation tips / story generation by Shufan Wang // [slides]

• Reading 1: Deep Learning for NLP Best Practices, Sebastian Ruder, 2017.
• Reading 2: Hierarchical Neural Story Generation, Fan et al., ACL 2018

Thu 11/12 - Constituency parsing // [slides]

• Optional reading: JM 13

Thu 11/14 - Guest lecture on linguistic probe tasks by Tu Vu // [slides]

• Reading 1: A Structural Probe for Finding Syntax in Word Representations, Hewitt and Manning, NAACL 2019.
• Reading 2: Do NLP Models Know Numbers? Probing Numeracy in Embeddings, Wallace et al., EMNLP 2019.
• Reading 3: Language Models as Knowledge Bases?, Petroni et al., EMNLP 2019.
• Optional reading 1: Designing and Interpreting Probes with Control Tasks, Hewitt and Liang, EMNLP 2019.
• Optional reading 2: Knowledge Enhanced Contextual Word Representations, Peters et al., EMNLP 2019.

Tues 11/19 - Semantic parsing // [slides]

• Optional reading 1: Learning to Map Sentences to Logical Form, Zettlemoyer & Collins, UAI 2005.
• Optional reading 2: Language to Logical Form with Neural Attention, Dong & Lapata, ACL 2016.

Thu 11/21 - Semantic parsing exercise

• HW 3 released, due Dec 5 on Gradescope // Colab notebook here

Dec 3 - Reading comprehension question answering // [slides]

• Optional reading 1: SQuAD: 100,000+ Questions for Machine Comprehension of Text. Rajpurkar et al., EMNLP 2016.
• Optional reading 2: Bidirectional Attention Flow for Machine Comprehension. Seo et al., ICLR 2017.

Dec 5 - Computational psycholinguistics // [slides]

• Class canceled because Mohit was sick :( Check out the slides if you're interested in what we would have covered!

Dec 10 - Final project poster presentations in CS Atrium from 1-4pm