Back To Home



A Neural Network for Factoid Question Answering over Paragraphs

Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daumé III
In Proceedings of EMNLP 2014, download PDF here

UPDATE (10/19/15): A much larger dataset that contains 20,407 questions about 2,347 answers has been released here (31 MB). This dataset includes questions about various categories (not just history and literature). For each answer in the dataset, we have also provided its corresponding Wikipedia page.

ABSTRACT: We introduce a recursive neural network model that is able to correctly answer paragraph-length factoid questions from a trivia competition called quiz bowl. Our model is able to succeed where traditional approaches fail, particularly when questions contain very few words (e.g., named entities) indicative of the answer. We apply our model, QANTA, to a dataset of questions from a trivia competition called quiz bowl. The model outperforms multiple baselines and, when combined with information retrieval methods, rivals the best human players.

CODE / DATA: Code for training and evaluating QANTA, along with two datasets1 that each contain around 2,000 question/answer pairs (~10,000 sentences), can be downloaded here (50 MB). See the provided README for usage instructions.

Please email any questions about the code and/or data to miyyer@umd.edu.

1Proprietary data from NAQT are excluded.

EXAMPLE:

QUESTION:
He left unfinished a novel whose title character forges his father's signature to get out of school and avoids the draft by feigning desire to join. A more famous work by this author tells of the rise and fall of the composer Adrian Leverkühn. Another of his novels features the jesuit Naptha and his opponent Settembrini, while his most famous work depicts the aging writer Gustav von Aschenbach. Name this German author of The Magic Mountain and Death in Venice.
ANSWER: Thomas Mann


The first sentence in this question contains no named entities or other words that are indicative of the answer by themselves. Nevertheless, QANTA is able to model the compositionality of the sentence in order to make the correct prediction. Below is a heatmap of the top five answers for the first sentence as scored by QANTA. Note how strongly indicative the plot description associated with the "novel" node is of the answer. Another interesting observation is that three of the top five answers are authors with well-known unfinished novels, which suggests that QANTA is able to learn information about the answer type without any manually-defined rules.


VECTOR SPACE VISUALIZATION:
Below is a t-SNE visualization of the 451 answers in our history dataset. The vector space is divided into six general clusters, and zooming in on each one in turn reveals interesting patterns. For example, the blue cluster is predominantly populated by U.S. presidents, and the zoomed plot depicts temporal clustering: presidents who were in office during the same time period occur closer together in the vector space.


ACKNOWLEDGMENTS:
We thank the anonymous reviewers, Stephanie Hwa, Bert Huang, and He He for their insightful comments. We thank Sharad Vikram, R. Hentzel, and the members of NAQT for providing our data. This work was supported by NSF Grant IIS-1320538. Boyd-Graber is also supported by NSF Grant CCF-1018625. Any opinions, findings, conclusions, or recommendations expressed here are those of the authors and do not necessarily reflect the view of the sponsor.

BIBTEX: 

			
				@inproceedings{Iyyer:Boyd-Graber:Claudino:Socher:Daume-2014,
				Title = {A Neural Network for Factoid Question Answering over Paragraphs},
				Author = {Mohit Iyyer and Jordan Boyd-Graber and Leonardo Claudino 
				and Richard Socher and Hal {Daum\'e III}},
				Booktitle = {Empirical Methods in Natural Language Processing},
				Year = {2014},
				Location = {Doha, Qatar},
				}