CS 690N, Spring 2018, UMass Amherst
Your in-class paper presentation will focus on one research paper,
and be about 20 minutes long.
Papers may cover any topic in NLP -- typically, something at the granularity of
a section of a textbook chapter might make sense.
If you need ideas, try looking at papers cited in
or the full 2nd edition book), which often have extensive bibliographies.
Eisenstein's draft textbook or other textbook or review sources.
Email me and Katie one to two weeks before your presentation with one to three
papers you are interested in presenting. We'll approve one or give you a
A few guidelines on paper selection, and paper research in general:
- Please use a "full" paper, typically at least 8 pages.
Or a journal paper that's longer!
- Anything from the major NLP conferences (ACL, NAACL, EMNLP) or journals (CL, TACL)
is certainly on topic. Others journals or conferences are certainly possible as well - like machine learning (NIPS, ICML, etc.), cognitive science or linguistics, or even general science (Science, PNAS, etc.).
The paper should talk about something regarding computation and language.
If it's just a machine learning paper that could be applied to anything that's not necessarily language, that's off-topic for this course.
- The paper can be new or old. If you present a newer paper, you should be prepared to give background about older work it builds off of.
If you'd like to do an older paper, it can be fun to randomly browse
the ACL Anthology, which goes all the way back
to the 1950s (!), though conference papers can get quite short and hard to say much about (the journals are better in this regard).
Things to make sure to explain in your presentation:
- What is the authors' research question? What problem are they trying to solve? Distill it to one sentence. Sometimes the paper authors do not concisely state this, so it is up to you to figure this out.
- Show an example of the NLP task, or type of data or linguistic phenomenon,
that the paper is about. Sometimes the paper authors do not do this, so it's up to you to explain it to the audience.
- Describe what the authors did, their results, and any implications it may have.
- Discussion: what questions did you have? What do you wish or want to be clarified? What do you think of the work and are there suggestions or ideas for future work?
This should be similar to discussion questions for the reading reactions.
For clarity and presentation style:
- Do not have more than 3-4 bullet points on one slide.
- Pictures and graphics can be much more effective than words.
- Have your slide titles be informative and convey the main message.
- Make sure you practice 2-3 times before giving the presentation!
You're giving a presentation to your fellow studens - make it worth our time
so that we will learn something.
- Here's some great advice: Michael Ernst, "How to give a technical presentation."
Presentations will be graded based on both content and clarity/presentation quality.
This is a great chance to practice
your presentation skills!
General tips for researching the literature
This may come in handy for finding an interesting paper to present,
as well as doing the literature review.
- Use Google Scholar or
to find related papers, and papers that cite a particular paper you've found.
- Always look at a paper's references list and try to find the most interesting-looking, or most-cited previous papers. Keep a text file or bookmarks of references that look promising, to check back on later.
- Learn the important terms in the research area, which will help find more
relevant or interesting papers -- for example, if you keep searching for
"chatbots", you might find there is a field called "dialog systems" which is
extremely relevant. Find the names of journals, conferences, and workshops in
the field, as well as particular people who do lots of resesarch in the area;
you can use all of this to find more related work.
- Make sure to skim papers when you first encounter them:
read the abstract and jump ahead to the results to roughly understand what they did. Decide later whether it's worth a deeper read.
- Search tip: for, say, NLP involving Wikipedia, try expanding with NLP or site keywords; e.g., "wikipedia nlp," "wikipedia site:aclweb.org," "wikipedia ACL", etc.
The literature review is a paper that reviews a subfield of NLP of your choice. It
must cover at least 15 resesarch papers, including at least 5 from before the year
(It can sometimes be harder to find older NLP papers that are also good/relevant.
But machine learning, statistics, and linguistics are substantially older disciplines
and nearly all NLP work builds on their ideas.)
We generally expect it to be 8-15 pages long (not inculding the references list at the end).
It should not merely describe the papers, but also synthesize, organize, and
relate them to one another and the broader literature in NLP, and ideally also ML
and linguistics. It can be done either individually or in a group of two.
Das and Martins (2007), "A Survey on Automatic Text Summarization"
(which itself was a class assignment, now highly cited!).
Another example, which has much more synthesis so isn't purely a literature review,
is the Turney and Pantel (2010) survey paper.
There are different ways to structure a literature review.
Typically, you should have something like:
- An introduction section that explains what the area is
and the motivation to study it --- why is it an interesting area of research, and why should the reader care?
- A main body, perhaps a few sections long, that describe previous work and specific research papers.
The most boring way to structure this is as a long list of papers
with a paragraph describing each. That's OK when you're writing notes for yourself,
but you should do some synthesis as well.
Try to group papers by common themes, methods, datasets, or assumptions.
What is similar and different among them?
Did the body of research change over the years?
Do different areas of research approach it differently?
A discussion and (potentially brief) conclusion,
which sums up the main points you made.
This may be a good place to discuss interesting possibilities
for future work,
or class projects!
Also make sure to:
- Use a reasonably sized font (say, 11pt) and margins.
The ACL stylesheet is optional but encouraged if you'd like to practice using it.
- Have a title, author names, and date.
- Have a properly and consistently formatted references list.
There are different standards to do this (for example,
opinions differ whether the page number is that relevant any more),
but make sure there is enough key information
for others to be able to find the paper,
know the exact version you are referring to,
and to make a quick determination of its credibility.
That means at least: authors, article title, name of publication or venue,
Here is a random sampling of papers that may be of interest,
either as themselves or as jumping off points for others.
Brown et al. 1993.
The Mathematics of Statistical Machine
Translation: Parameter Estimation.
Grosz et al. 1995.
Centering: A Framework for Modeling the
Local Coherence of Discourse.
Pang et al., 2002.
Thumbs Up? Sentiment Classification Using Machine Learning Techniques.
Proceedings of EMNLP.
Discriminative Training Methods for Hidden Markov Models:
Theory and Experiments with Perceptron Algorithms.
Proceedings of EMNLP.
Sauri and Pustejovsky, 2012.
Are You Sure That This Happened?
Assessing the Factuality Degree of
Events in Text.
Tsvetkov and Dyer, 2016.
Cross-Lingual Bridges with Models of Lexical Borrowing.
Chen et al. 2016.
A Thorough Examination of the
CNN/Daily Mail Reading Comprehension Task.
Proceedings of ACL.
What to do about bad language on the internet.
Proceedings of NAACL.
Caliskan et al. 2017.
Semantics derived automatically from language corpora contain human-like biases.
Ramiro et al.,
Algorithms in the historical emergence of word senses.
Possibly of interest: these ACL Anthology pages let you see number citations of papers for entire venues; you can rank by citation count (it only tracks within ACL Anthology papers) to see popular ones. They're not always interesting, but are sometimes.
Papers on text analysis as a tool for social science and the humanities:
Several papers from the Journal of Digital Humanities, 2(1).
To start, see the overview:
Weingar and Meeks, 2012.
The Digital Humanities Contribution to Topic Modeling.
Grimmer and Stewart, 2013.
Text as Data: The Promise and Pitfalls of Automatic Content
Analysis Methods for Political Texts.
Monroe et al., 2008.
Fightin’ Words: Lexical Feature Selection and
Evaluation for Identifying the Content of Political
One of the social science-oriented papers from
the webpage for Structural Topic Models.