Introduction to Natural Language Processing

Computer Science 585 — Fall 2009



Lectures:Tuesdays, Thursdays 4-5:15, Engineering Lab 305
Instructor:David Smith (
TA:Jason Naradowsky (
Office hours:Smith: W 1:30-3:30, Th 5:15-6:15, or by appointment, in CS Building 358
Naradowsky: T 5:15-6:15, W 1-2, in CS Building 264
Class email

Texts and Toolkits

Manning and Schütze's excellent Foundations of Statistical Natural Language Processing is available online from the UMass library or in stores.

Bird, Klein, and Loper's Natural Language Processing with Python is available at the NLTK site.


Tue Sep 8Course Overview: Why NLP? [slides]
Thu Sep 10The Chomsky Hierarchy, Grammars, and Hand-Crafted NLP
[McCallum's slides on regular expressions]
Manning & Schütze, chapter 3Homework #1 due Sep. 24
Tue Sep 15Counting Things: Text Classification and N-gram Models [slides]M&S 2.1
Thu Sep 17Counting Things (cont'd): Text Classification and N-gram Models [slides]M&S 6
Tue Sep 22No Class: CS Graduate Research Methods Seminar
Thu Sep 24Estimation: Maximum Likelihood and Smoothing [slides]Homework #1 due
Tue Sep 29Hidden Markov Models: Inferring Hidden Tags and Segmentations [slides]M&S 9
Thu Oct 1Hidden Markov Models: Message Passing with the Viterbi and Forward-Backward Algorithms [slides]
Tue Oct 6Context-Free Grammars: Inferring Hidden Trees [slides]M&S 11Homework #2 assigned, due Oct. 15
Thu Oct 8Context-Free Parsing: Chomsky Normal Form, the CKY Algorithm, and Probabilistic Parsing [slides]
Tue Oct 13Context-Free Parsing: Outside Computations and Earley's Algorithm [slides] (Holiday: makeup for Sep. 22)
Thu Oct 15Context-Free Parsing: Adding Features and Improving Efficiency [slides]M&S 12Homework #2 due
Tue Oct 20Practical Parsing: Lexicalized Models and Algorithms [slides]
Thu Oct 22Midterm
Tue Oct 27Log-Linear (Maximum Entropy) Models and Conditional Likelihood [slides]NLTK ch. 6
Thu Oct 29Log-Linear Models with Structured Outputs [slides]
Tue Nov 3Log-Linear Models with Structured Outputs (with examples from dependency parsing) [slides]
Thu Nov 5Semantics [slides]NLTK ch. 10
Tue Nov 10Computational Semantics [slides]
Thu Nov 12Lexical Semantics [slides]M&S 5, 7
Tue Nov 17Lexical Semantics 2 [slides]
Thu Nov 19Machine Translation: Overview and Word Alignment [slides]
Tue Nov 24Machine Translation: Non-Parallel Text [slides]
Thu Nov 26Holiday: Thanksgiving
Tue Dec 1Machine Translation: Searching for Good Translations [slides]
Thu Dec 3Machine Translation: Training [slides]
Tue Dec 8Information Extraction and Question Answering [slides]NLTK ch. 7Homework #3 assigned
Thu Dec 10NLP and Linguistics [slides]
Tue Dec 15Review Session: same time, same place [slides]
Wed Dec 1610:30 a.m., Computer Science Building 142: Final Exam