Schedule

Homeworks

Weekly topics and readings.

"Goldberg (2017) book draft" is on Piazza here.
"Eisenstein (2017) draft notes" is on Github here.

Topic	Readings
1/24, 1/26: Language modeling and EM HW1	Pereira (2000), Formal grammar and information theory: together again? Ch 1, Ch 4.1-4.3 and Ch. 5, Eisenstein (2017) draft notes (Ch 5 on LMs, and 4.2/4.3 on EM; and Ch 1 on probability+Naive Bayes) Saul and Pereira (1997) Whiteboard on 1/31: EM in 2 pages
1/31, 2/2 (and 2/7): Log-linear models	Rosenfeld (1996) Collins (lecture notes, unknown date), "Log-linear Models" Goldberg (2017) book draft, Ch 6-7 on features
2/14, 2/16: Neural models	5.3, Eisenstein (2017) draft notes, "Recurrent NN LMs" Goldberg (2016), "Primer on NNs for NLP," JAIR Optional: Goodfellow et al. (2016), deeplearningbook.org. Oxford/DeepMind 2017 course on DL/NLP. Bengio et al. 2003 for the Markov NN language model. High-level perspectives in Bengio et al. (2012), "Representation Learning" and Manning (2016), "Computational Linguistics and Deep Learning". Pop science storytelling in New York Times (Dec. 2016) on the development of Google's impressive neural MT system (the article gives short shrift to the 20 years of machine learning for MT before it, of course)
2/21, 2/23, 2/28: Sequence Labeling	Ch. 6 and 7, Eisenstein (2017) draft notes. Some HMM notes and HMM handout Sarawagi and Cohen (2004): Semi-Markov CRFs
3/2, 3/7, 3/9: Syntax, CFGs	Ch. 3 from Manning and Schuetze (1999): "Linguistic Essentials." Ch. 10 and 11, Eisenstein (2017) notes, "CFGs" and "CFG Parsing." Optional: Stefan Mueller (2015), "Grammatical theory: From transformational grammar to constraint-based approaches." Overviews many different approaches.
Spring break
3/21, 3/23: Dependency Parsing	Eisenstein 2017, Chapter 12, "Dependency Parsing" A few applications: Lin and Pantel 2001, Erkan et al. 2007, Bamman et al. 2013, Goldberg and Orwant 2013
3/28: Coreference	Eisenstein 2017, Chapter 17, Anaphora and Coreference Resolution Optional: de Marneffe et al. 2015, "Modeling the Lifespan of Discourse Entities with Application to Coreference Resolution"
3/30: Relational semantics	Eisenstein 2017, Chapter 13+14, Logical and Shallow semantics (we'll emphasize shallow) Hobbs et al. 1997, "FASTUS: A Cascaded Finite-State Transducer for Extracting Information from Natural-Language Text"
4/4: Guest lecture, Brian Dillion, computational models of human sentence comprehension. Notes
4/6: Distributional semantics	Eisenstein 2017, Chapter 15, Distributional semantics
4/11, 4/13: Structured neural networks (convolutional, recurrent, recursive)	Karpathy (2015) blog post: The Unreasonable Effectiveness of Recurrent Neural Networks Sections 9-13 of Goldberg (2016), "Primer on NNs for NLP," JAIR Linzen et al. (2016), Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies Socher et al. (2013), Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank Optional for many more details: Chapter 10 of Goodfellow et al. (2016), Sequence Modeling: Recurrentand Recursive Nets
4/18: No class (Monday schedule day)
4/20: Autodiff demo	General review: Baydin et al. 2015 DyNet paper, on issues for structured NLP modeling (PyTorch has similar architecture (?)) Stan paper Software Dynamic libraries: PyTorch, DyNet, Chainer, Autograd. Static: Tensorflow, Theano. Applications - neural networks: Keras, Lasagne, Caffe, Caffe2, etc. Applications - statistical modeling: Stan (compare to BUGS)
4/20: Crowdsourcing demo	Many resources from Callison-Burch, Ungar and Pavlick's 2015 tutorial and a related course at crowdsourcing-class.org. And there's always Snow et al. 2008
4/25: Interpretable BOW models (topic models, sparse log-linear) and Bayesian models	(Sparse) additive logit models Eisenstein et al. 2011, Sparse Additive Generative Models of Text Monroe et al. 2008, Fightin Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict Longer but interesting: Taddy 2013, Multinomial Inverse Regression for Text Analysis Topic models Roberts et al. 2013, The Structural Topic Model and Applied Social Science (see also longer papers on the STM website) Boyd-Graber et al. 2014, Care and Feeding of Topic Models
4/27: Dialects and social media NLP (Su Lin Blodgett)	Blodgett et al. 2016, Demographic Dialectal Variation in Social Media: A Case Study of African-American English
5/2: Presentations