Advanced Topics in Natural Language Processing

CS 685, Spring 2021, UMass Amherst

Instructor: Brendan O'Connor
TA: Nader Akoury
Lectures: MW 2:30PM - 3:45PM, online on Zoom
Office Hours (US Eastern Time), see Slack #general channel for meeting links


Please contact Nader if you are having issues accessing any of the resources.

Course Information

Description: This course covers a broad range of advanced level topics in natural language processing. It is intended for graduate students in computer science who have familiarity with machine learning fundamentals, and previous course or research experience in natural language processing. It may also be appropriate for computationally sophisticated students in linguistics and related areas. Topics include probabilistic models of language, computationally tractable linguistic representations for syntax and semantics, neural network models for language, and selected topics in discourse and text mining. After completing the course, students should be able to read and evaluate current NLP research papers. Coursework includes a research literature review, homework assignments, and a final project.

Prerequisites: Familiarity with multivariate calculus, probability theory, dynamic programming, and mathematical foundations of machine learning algorithms, such as from COMPSCI 688 (Probabilistic Graphical Models), 689 (Machine Learning), or equivalent. COMPSCI 682 (Deep Learning) will be helpful, but not required. Experience in linguistics will also be helpful, but not required.

Course topics

CS685 is an Advanced Topics course, with different focuses each semester. This semester, we will focus on

Related courses

See also previous versions or variants of this course: 685 in Fall 2020, 685 in Spring 2020, 690D in Spring 2019, and 690N in Spring 2018. See also this list of courses in NLP and related areas offered at UMass and nearby.


We'll use the Eisenstein text for many of the readings in this course.

In general there are a number of useful texts on NLP, as well as important adjacent areas like linguistics and machine learning. For some of the links below, you need to sign in to the UMass VPN (or access from campus) to get access via the UMass Library subscription.

Note some of these authors were recently lamenting the lack of sociolinguistics in the usual NLP curriculum, and its lack of presence in their own books! There is much to learn and intellectual change in progress.

Math review

Calculus, linear algebra, and probability theory are the required mathematical background for the course. For review, especially of the HW0 topics, the following resources may be helpful.

Courses elsewehre

Georgetown, Stanford, Johns Hopkins, Georgia Tech, U Texas, Berkeley, Coursera (discontinued?), etc.