Homeworks

HW1.ipynb (word statistics): due Friday 9/16 at midnight. (also in HTML format)

HW2.zip (n-gram LMs): due Friday 9/23 at midnight. (if desired, see HTML format)

HW3.pdf (Naive Bayes): due Friday 10/7 at midnight. Also get nb.py (starter code), and the data file, which is around 46 MB to download, and takes around 210 MB when unzipped.

HW4: this is divided into two parts. The homework is much more substantial than previous homeworks. Please get started early.

• HW4 Part (A): HMM and Viterbi. Due Friday 10/21 at midnight. vit.py starter code.
• HW4 Part (B): Structured perceptron tagger. Due Friday 10/28 at midnight. structperc.py starter code, and oct27.train and oct27.dev datasets.

HW5.zip (distributional similarity): due Sunday 11/27. (also in HTML format)

HW6.pdf (research paper reading): due Friday 12/9.

Schedule

Make sure to reload this page to ensure you’re seeing the latest version.

Readings should be done before the indicated class.

Tu 9/6 - Introduction [slides pdf]

• HW0 is out. Due Friday 9/9 at 5pm on Moodle. (hw0.tex if you want it). Solutions on the Piazza "Resources" section.

Th 9/8 - Words

• Reading: JM 3rd ed, 2.1-2.3 (from the "Regex, Text Normalization, Edit Dist" chapter)
• See also: "Unix for Poets" by Ken Church
• Lecture materials: partial slides, scanned slides, jupyter demo HTML, jupyter demo.ipynb file, grep commands

Tu 9/13 - N-gram Language Models [slides]

• JM 3rd ed, 4.1-4.4
• Optional: Peter Norvig, Natural Language Corpus Data

Th 9/15 - N-gram Language Models, cont'd [slides]

• Exercise 1, Shannon game: PDF, results

Tu 9/20 - Naive Bayes Classification [slides: 1st half of this pdf]

• JM 3rd ed, 7.1-7.3

Th 9/20 - Log-linear Classification [slides: 2nd half of this pdf]

• JM 3rd ed, 7.4-7.6

Tu 9/27 - Sentiment Lexicons (+ very brief neural networks intro) [slides]

• JM 3rd ed, Ch. 21
• Optional (or use for Exercise 2 alternative if can't make Bengio talk): Manning 2016, "Computational Linguistics and Deep Learning"

Th 9/29

Lecture is cancelled. Instead, go to the Yoshua Bengio talk in the Statistical and Computational Data Science Distinguished Lecture Series. 2-4pm, CS Building room 150/151. Talk video here. The makeup reading is Manning 2016, "Computational Linguistics and Deep Learning".

Tu 10/4: Part-of-speech Tags [POS slides] [Project slides]

• JM 3rd ed, 9.1-9.3 from Ch. 9, "Part-of-Speech Tagging."
• In-class Exercise 3: part-of-speech tags

Th 10/6: Hidden Markov Models, Forward Algo [slides]

• JM 3rd ed, Ch. 8 "Hidden Markov Models", plus 9.4 from POS chapter

No class on Tuesday 10/11

Th 10/13: Viterbi Algorithm [slides] [scans]

• JM 3ed, 8.4 "The Viterbi Algorithm"
• In-class Exercise 4: Viterbi

Friday 10/14: Project proposals due

Tu 10/18: Structured Perceptron and Conditional Random Fields [slides]

• Daume, CIML, 3.1-3.3 and 3.6 on the perceptron algorithm and averaged perceptron training. Note Daume's version is only for binary classification; our multiclass/structured setting is slightly different.
• Chen (2012)'s blogpost on CRFs. Note that we won't use gradient-based learning.
• Optional: 9.1-9.3 of Eisenstein (2016) may be helpful.
• Optional: Sutton and McCallum's CRF tutorial may be helpful; we've had issues in the past with one or two of its equations.

Th 10/20: CRFs part 2 and project work

• Slide scans from today. Also I wrote a CRF worked example after class (one big page, two pages versions)

10/25: Syntax, Part 1 [slides]

• Exercise 5: Noun phrase CFG
• Reading: from J&M 2nd ed., 12.1-12.7 from Ch. 12, "Formal Grammars of English." This is a different edition than what we've used for the other readings. See the Piazza resources page.
• Optional: Phillips (2003) is a great overview of syntax.

10/27: Syntax, Part 2 [slides]

• Exercise 6: CKY
• Reading: from J&M 2nd ed., Ch. 13, "Parsing with Context-Free Grammars." Again see the Piazza resources page.

Tu 11/1: Review session

• Practice midterm questions
• Pracice midterm, partial solutions

Th 11/3: Midterm

11/8: Lexical Semantics [slides]

• JM 3rd ed., Ch. 17, Computing with Word Senses (only covered 17.1-17.3)

11/10: Distributional Semantics [slides]

• JM 3rd ed: Ch. 15 (Vector Semantics) and Ch. 16 (Semantics with Dense Vectors)

Sunday 11/13: Progress report due

11/15: Word Embeddings and Neural Networks [slides]

• Tensorflow web demo

11/17: Relationship Extraction, Neural Networks, and Matrix Factorization [slides]

No class 11/22 or 11/24: Thanksgiving

11/29: Machine Translation [slides]

• Exercise 7: alignments
• J&M 2nd edition, 25.1-25.5

(Lecture was replaced): EM and latent variable models

• EM for IBM Model 1: J&M 2nd edition, 25.6
• EM for a latent-variable Markov model: Saul and Pereira (1997), "Aggregate and mixed-order Markov models for statistical language processing"
• Optional (EM to refute Chomsky): Pereira (2000), "Formal grammar and information theory: together again?"
• Optional: Blei (2012), "Probabilistic Topic Models"

12/1: Information Extraction and Coreference [slides]

12/6: Research and summarization (Abe)

12/8: Social factors and ethics in NLP

• Hovy and Spruit (2016), "Beyond Privacy: The Social Impact of Natural Language Processing"
• Blodgett et al. (EMNLP 2016), "Demographic Dialectal Variation in Social Media: A Case Study of African-American English"

Tu 12/13: Poster session, 2:30-4:30, room CS 150/151