CS490A: Applications of Natural Language Processing

CS 490A, Fall 2020, UMass Amherst CS
Lecture: TTh 2:30-3:45PM, on Zoom

Instructor: Brendan O'Connor
TAs: Yuanguo Lang, Rumeng Li, and Tomas Geffner
To contact us: use Piazza private post, or use the instructors' email here.

Links:

• Schedule
• Grading / Policies
• Projects
• Gradescope for assignment submission
• Piazza "logistics" post for Zoom link and office hours.
• Piazza "Resources" page for homeworks and Zoom lecture videos.

Course description

Natural Language Processing (NLP) is the engineering art and science of how to teach computers to understand human language. NLP is a type of artificial intelligence technology, and it's now ubiquitous -- NLP lets us talk to our phones, use the web to answer questions, map out discussions in books and social media, and even translate between human languages. Since language is rich, ambiguous, and very difficult for computers to understand, these systems can sometimes seem like magic -- but these are engineering problems we can tackle with data, math, and insights from linguistics.

This course will introduce NLP methods and applications including probabilistic language models, machine translation, and parsing algorithms for syntax and the deeper meaning of text. During the course, students will (1) learn and derive mathematical models and algorithms for NLP; (2) become familiar with key facts about human language that motivate them, and help practitioners know what problems are possible to solve; and (3) complete a series of hands-on projects to implement, experiment with, and improve NLP models, gaining practical skills for natural language systems engineering.

This course is intended for upper-level CS undergraduates, as well as linguistics students with an appropriate computational background.

Prerequisites: experience in programming and probability. Undergraduates must have completed:
  (CS220 and CS240) or Ling492B
This is intended to represent the following requirements:

1. Programming maturity, including data structures and recursion (e.g. comfort with implementing/debugging depth-first search).
2. Basic algorithm analysis (e.g. big-O analysis of a graph algorithms).
3. Basic probability theory (e.g. Bayes Rule).
4. A genuine interest in language; linguistics background is a huge plus.

Related courses

See also previous offerings of the similar CS 585 course.

If you are interested in other courses, see this list of courses in NLP and related areas offered at UMass and nearby.

Readings

• The main suggested textbook is Jurafsky and Martin, Speech and Language Processing, 3rd ed. Draft PDFs. We will post readings from it.
• Also useful is Eisenstein, Natural Language Processing. Pre-print PDF (book also available). We will have some readings from it.

Other useful texts for NLP include:

• Bender, Linguistic Fundamentals for NLP. Short book. Focuses on linguisic issues relevant to NLP.
• Manning and Schütze, Foundations of Stat NLP. Free access at UMass.
• Smith, Linguistic Structure Prediction. Free access at UMass. Short book. Structured prediction inference methods for NLP.
• Murphy, Machine Learning: a Probabilistic Perspective. Excellent, though advanced, coverage of most of the machine learning methods we will use.
• Bird et al, NLP with Python, a.k.a. the NLTK book. Aimed at a more introductory level than this course, but the book is a good gentle introduction to NLP with a CL (computational linguistics) emphasis. The NLTK software has easy-to-use data access and some interfaces to (not always SOTA) NLP tools.