CS 690N, Spring 2018, UMass Amherst

Presentations

Papers may cover any topic in NLP -- typically, something at the granularity of a section of a textbook chapter might make sense. If you need ideas, try looking at papers cited in the Jurafsky+Martin textbook (draft chapters, or the full 2nd edition book), which often have extensive bibliographies. Also see Eisenstein's draft textbook or other textbook or review sources.

Email me and Katie one to two weeks before your presentation with one to three possible papers you are interested in presenting. We'll approve one or give you a different one.

A few guidelines on paper selection, and paper research in general:

• Please use a "full" paper, typically at least 8 pages. Or a journal paper that's longer!
• Anything from the major NLP conferences (ACL, NAACL, EMNLP) or journals (CL, TACL) is certainly on topic. Others journals or conferences are certainly possible as well - like machine learning (NIPS, ICML, etc.), cognitive science or linguistics, or even general science (Science, PNAS, etc.). The paper should talk about something regarding computation and language. If it's just a machine learning paper that could be applied to anything that's not necessarily language, that's off-topic for this course.
• The paper can be new or old. If you present a newer paper, you should be prepared to give background about older work it builds off of. If you'd like to do an older paper, it can be fun to randomly browse the ACL Anthology, which goes all the way back to the 1950s (!), though conference papers can get quite short and hard to say much about (the journals are better in this regard).

Things to make sure to explain in your presentation:

• What is the authors' research question? What problem are they trying to solve? Distill it to one sentence. Sometimes the paper authors do not concisely state this, so it is up to you to figure this out.
• Show an example of the NLP task, or type of data or linguistic phenomenon, that the paper is about. Sometimes the paper authors do not do this, so it's up to you to explain it to the audience.
• Describe what the authors did, their results, and any implications it may have.
• Discussion: what questions did you have? What do you wish or want to be clarified? What do you think of the work and are there suggestions or ideas for future work? This should be similar to discussion questions for the reading reactions.

For clarity and presentation style:

• Do not have more than 3-4 bullet points on one slide.
• Pictures and graphics can be much more effective than words.
• Have your slide titles be informative and convey the main message.
• Make sure you practice 2-3 times before giving the presentation! You're giving a presentation to your fellow studens - make it worth our time so that we will learn something.
• Here's some great advice: Michael Ernst, "How to give a technical presentation."

Presentations will be graded based on both content and clarity/presentation quality. This is a great chance to practice your presentation skills!

General tips for researching the literature

This may come in handy for finding an interesting paper to present, as well as doing the literature review.

• Use Google Scholar or Semantic Scholar to find related papers, and papers that cite a particular paper you've found.
• Always look at a paper's references list and try to find the most interesting-looking, or most-cited previous papers. Keep a text file or bookmarks of references that look promising, to check back on later.
• Learn the important terms in the research area, which will help find more relevant or interesting papers -- for example, if you keep searching for "chatbots", you might find there is a field called "dialog systems" which is extremely relevant. Find the names of journals, conferences, and workshops in the field, as well as particular people who do lots of resesarch in the area; you can use all of this to find more related work.
• Make sure to skim papers when you first encounter them: read the abstract and jump ahead to the results to roughly understand what they did. Decide later whether it's worth a deeper read.
• Search tip: for, say, NLP involving Wikipedia, try expanding with NLP or site keywords; e.g., "wikipedia nlp," "wikipedia site:aclweb.org," "wikipedia ACL", etc.

Literature Review

The literature review is a paper that reviews a subfield of NLP of your choice. It must cover at least 15 resesarch papers, including at least 5 from before the year 2000. (It can sometimes be harder to find older NLP papers that are also good/relevant. But machine learning, statistics, and linguistics are substantially older disciplines and nearly all NLP work builds on their ideas.)

We generally expect it to be 8-15 pages long (not inculding the references list at the end).

It should not merely describe the papers, but also synthesize, organize, and relate them to one another and the broader literature in NLP, and ideally also ML and linguistics. It can be done either individually or in a group of two.

An excellent example is Das and Martins (2007), "A Survey on Automatic Text Summarization" (which itself was a class assignment, now highly cited!). Another example, which has much more synthesis so isn't purely a literature review, is the Turney and Pantel (2010) survey paper.

There are different ways to structure a literature review. Typically, you should have something like:

• An introduction section that explains what the area is and the motivation to study it --- why is it an interesting area of research, and why should the reader care?
• A main body, perhaps a few sections long, that describe previous work and specific research papers. The most boring way to structure this is as a long list of papers with a paragraph describing each. That's OK when you're writing notes for yourself, but you should do some synthesis as well. Try to group papers by common themes, methods, datasets, or assumptions. What is similar and different among them? Did the body of research change over the years? Do different areas of research approach it differently?
• A discussion and (potentially brief) conclusion, which sums up the main points you made. This may be a good place to discuss interesting possibilities for future work, or class projects!

Also make sure to:

• Use a reasonably sized font (say, 11pt) and margins. The ACL stylesheet is optional but encouraged if you'd like to practice using it.
• Have a title, author names, and date.
• Have a properly and consistently formatted references list. There are different standards to do this (for example, opinions differ whether the page number is that relevant any more), but make sure there is enough key information for others to be able to find the paper, know the exact version you are referring to, and to make a quick determination of its credibility. That means at least: authors, article title, name of publication or venue, date.

Suggested Papers

Here is a random sampling of papers that may be of interest, either as themselves or as jumping off points for others.

• Brown et al. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics.
• Grosz et al. 1995. Centering: A Framework for Modeling the Local Coherence of Discourse. Computational Linguistics.
• Pang et al., 2002. Thumbs Up? Sentiment Classification Using Machine Learning Techniques. Proceedings of EMNLP.
• Collins, 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. Proceedings of EMNLP.
• Sauri and Pustejovsky, 2012. Are You Sure That This Happened? Assessing the Factuality Degree of Events in Text. Computational Linguistics.
• Tsvetkov and Dyer, 2016. Cross-Lingual Bridges with Models of Lexical Borrowing. JAIR.
• Chen et al. 2016. A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task. Proceedings of ACL.
• Eisenstein 2013. What to do about bad language on the internet. Proceedings of NAACL.
• Caliskan et al. 2017. Semantics derived automatically from language corpora contain human-like biases. Science.
• Ramiro et al., 2018. Algorithms in the historical emergence of word senses. PNAS.

Possibly of interest: these ACL Anthology pages let you see number citations of papers for entire venues; you can rank by citation count (it only tracks within ACL Anthology papers) to see popular ones. They're not always interesting, but are sometimes.

Papers on text analysis as a tool for social science and the humanities:

• Several papers from the Journal of Digital Humanities, 2(1). To start, see the overview: Weingar and Meeks, 2012. The Digital Humanities Contribution to Topic Modeling.
• Grimmer and Stewart, 2013. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis.
• Monroe et al., 2008. Fightin’ Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict. Political Analysis.
• One of the social science-oriented papers from the webpage for Structural Topic Models.

Other areas.

• Look at the "Workshops" line on the ACL Anthology. Workshops tend to have topically focused sets of papers. Most recent ones also have external web sites. For example:
  • First ACL Workshop on Ethics in Natural Language Processing
  • Workshops on NLP and educational applications, such as BEA11.
• Computational humor
• Stylometry
• Many more!