Homework assignments

See Gradescope for the most up to date due dates.

• HW0 (math review)
• HW1 (tokenization and Naive Bayes). Links: hw1.py, hw1.sh, lotr_script.txt, movie review dataset (44 MB) (for convenience: html notebook)
• HW2 (annotation and modeling experiment). Datasets.
• HW3, HW3(tex file)
• HW4

Course topics outline

• Words and Classification Models (9/7-9/21)
• Datasets and Classification Applications (9/23-10/7)
• Linguistic Structure (10/12-10/21, approx.)
• Embeddings, Neural Networks (10/26-11/2, approx.)
• BERT and Pre-trained LMs (11/16-11/18, approx.)
• NLP Applications (11/23-12/2, approx.)

Schedule

Make sure to reload this page to ensure you’re seeing the latest version. Readings should be done before the indicated class. The main textbooks are available free online:

• JM = Jurafsky and Martin, Speech and Language Processing, 3rd edition. (We'll use this one the most.)
• INLP = Eisenstein, Intro to NLP, online text (or book version)

Tentative schedule follows.

Thu 9/2: Introduction

Tue 9/7: Words, regexes, tokenization

• Reading: JM ch. 2.1, 2.2, 2.4
• Reading: David C. Zentgraf, "What Every Programmer Absolutely, Positively Needs To Know About Encodings And Charater Sets To Work With Text"
• Optional: Ken Church, "Unix for Poets"
• Optional: RegexOne

Thu 9/9: Word probs and N-gram language models

• Exercise 2
• Word counts demo html ipynb
• Reading: JM ch. 3.1-3.4

Tue 9/14: Naive Bayes Text Classification

• Slides for 9/14 and 9/16.
• Also continue N-gram LMs.
• Reading: JM 4.1-4.6
• Optional: INLP 4.1-4.3

Thu 9/16: Naive Bayes Text Classification, continued

• In-class Exercise 3: Naive Bayes example
• In-class manual keyword sentiment demonstration (ipynb)
• Readings on evaluation (we didn't get to this in class)
  • Reading: JM 4.7-4.10
  • Optional: INLP ~4.4

Tue 9/21: Logistic Regression Classification

• Slides, Annotated Slides
• Reading: JM ch. 5

Thu 9/23: Annotation and Labeling

• Annotated slides/notes from lecture. Includes model pros/cons, evaluation, overfitting/regularization.
• Reading: INLP 4.5
• In-class Exercise 4: Interannotator agreement

Tue 9/28: On Data - Collection and Creation

• Reading: Gebru et al., "Datasheets for Datasets"
• Reading: Krause, "Data Biographies"

Thu 9/30: On Data - Considerations

• Excercise 5
• Reading: Dastin, "Amazon scraps secret AI recruiting tool that showed bias against women"
• Reading: Wilson, "The Unknown Perils of Mining Wikipedia"

Tue 10/5: Classification Applications

• Annotated Slides
• Reading (review): Figures 7-10 (pp. 21-24) from Gebru et al., "Datasheets for Datasets"
• Optional: Pang et al., "Thumbs up? Sentiment Classification using Machine Learning Techniques"
Note: What this paper calls "maxent" we now call "logistic regression". The differences between logistic regression and linear SVMs are now typically minor.

Thu 10/7: Classification Applications, Final Project Discussion

• Exercise 6
• Reading: Long & So, "Literary Pattern Recognition"

Tue 10/12: Tagging, part 1: Parts of Speech and Named Entities

• Exercise 7, POS tags
• Lecture notes scan
• Reading: JM 8.1-8.3
• Reading: Justeson and Katz (1995)

Thu 10/14: Tagging, part 2

• No separate readings for this day. Topics include: lexical ambiguity, word sense disambiguation, NER, BIO encoding, and sequence models.

Tue 10/19: Grammars and Syntactic Parsing

• Lecture notes scan
• Exercise 8, CFGs
• Reading: JM 12.1-12.4 on constituency grammars
• Reading: JM 14.1-14.3 on dependency parsing

Thu 10/21: Syntactic Parsing Applications

• Annotated Slides
• No separate readings for this day.

Tue 10/26: Word Embeddings, part 1

• Reading: JM ch. 6

Thu 10/28: Word Embeddings, part 2

• Lectures notes scan
• Word embedding demo from class
• Reading: JM ch. 6
• "Exercise 9" due tomorrow: schedule your project meeting with your TA
• Exercise 10 on word embeddings (due after midterm)
• 11/2 office hours: dependency parsing and cosine similarity

Tue 11/2: Neural Networks in NLP

• Reading: JM 7.1-7.4 (note: reading was not posted before lecture)

Thu 11/4: Midterm Review and Misc. Topics

• Practice midterm questions, annotated questions from lecture

Tue 11/9: In-class midterm

No class Thu 11/11 (Veteran's Day)

Tue ll/16: Contextualized Word Representations

• Annotated Slides
• Optional: Bloem, "Transformers from Scratch"

11/18: BERT and Beyond

• Annotated Slides
• Exercise 11
• Optional: Rogers et al., "A Primer in BERTology"

Tue 11/23: Project work day and poster session info

Tue 11/30: Statistical Testing in NLP

• INLP 4.4.3-4.4.4

Thu 12/2: Topic Models

• Reading: Boyd-Graber et al. Applications of Topic Models, Ch. 1
• Optional: Jockers, "LDA Buffet"
• Optional: Schmidt & Frass, "The Language of The State of The Union"
  Note: Not on topic models, but useful to compare this article's analysis with your trained topic model from the lecture activity.

Tue 12/7: Final presentations

Last day of class! The poster or presentation session will take place during and around the standard lecture time, on GatherTown! See the 11/23 slides.

• Exercise 12

Thu 12/16: Final project due (last day of finals)

Final project reports are due by the last day of the final exam period. Late days cannot be used.