Nicholas Monath

Research Scientist
Google
nicholas.monath@gmail.com

Bio

I am a research scientist at Google. I defended my PhD in computer science in 2021 at the University of Massachusetts Amherst advised by Professor Andrew McCallum, where I was a member of the Information Extraction and Synthesis Laboratory.

I have had the great pleasure of collaborating with Ari Kobren (Oracle), Manzil Zaheer (Google), Avinava Dubey, Amr Ahmed (Google), Akshay Krishnamurthy (MSR), Sunil Mohan (CZI), Ivana Williams (CZI), Sebastian Macaluso (NYU, Physics), Kyle Cranmer (NYU, Physics), Pat Flaherty, Rajarshi Das (UMass), Rico Angell (UMass), Nishant Yadav (UMass), Mrinmaya Sachan (UMass), and more (see papers below)!

During my PhD, I am grateful for the oppurtunity to have interned at: Google (Winter 2018-2019, 2019-2020) with Amr Ahmed, Manzil Zaheer and Avinava Dubey; CZI (Summer 2019) with Ivana Williams; IBM (Summer 2018) with Michael Glass and Alfio Gliozzo, and Amazon (Summers 2016, 2017) with Shankar Ananthakrishnan and Bo Xiao.

I received my B.S. in computer science and mathematics from Brandeis University in 2013 where I worked with Professors James Storer and Antonella Di Lillo.

Research Interests

I work on machine learning and natural language processing. In particular, I am most interested in scalable clustering, incremental/online clustering, and entity discovery and linking.

Discovering Clusters in Continuously Arriving Data. Often we are faced with settings in which new data that arrives all of time. Performing clustering in these settings requires both the discovery of new (emerging) clusters as well as the reconsideration of past clustering decisions in the presence of newly arrived data. We have been designing algorithms that use hierarchical structured clusterings to reperesent uncertainty in this incremental setting and use tree rearrangements to efficiently reconsider past clustering decisions. Related publications: KDD 2019, KDD 2017.

Entity Discovery. Entities are mentioned ambiguously in both structured (e.g., databases) as well as unstructured (e.g., natural language) data. These entities may belong to many different (possibly overlapping) types. Resolving the ambiguity present in these entity mentions and/or discovering their entity types is necessary for many tasks such as the automatic construction of knowledge-bases, and question answering. This task is especially difficult when the catalog of entities or attributes are not known in advance. This last problem, often known as “entity discovery,” is one of clustering, with the special challenges that it needs to operate on incrementally arriving data, at large scale, ideally with non greedy behavior. Related publications: Related publications: NAACL 2021, Findings of EMNLP 2020, KDD 2019, ACL 2019.

Modeling Abstractions for Continual / Life-long Learning. Clusters provide summaries of data points. Hierarchical and DAG-structured clustering can provide abstractions at multiple granularities. Recent work in continual and few shot learning has shown that we can use clusters in place of individual points to more effectively generalize to new classes as well as to prevent catastrophic forgetting. Related publications: Findings of EMNLP 2020.

Relaxing Tree Structures. We are interested in alternatives to standard hierarchical clustering representations to facilitate end-to-end optimization, richer representations of uncertainty, and scalability. We have proposed continuous representations of trees in the unit ball as well as DAG-structured alternatives to trees. Related publications: AISTATS 2021, KDD 2019.

Conference Publications

Tianyu Liu, Yuchen Eleanor Jiang, Nicholas Monath, Ryan Cotterell, Mrinmaya Sachan.
Autoregressive Structured Prediction with Language Models
. Findings of EMNLP, 2022. [pdf]
Nishant Yadav, Nicholas Monath, Rico Angell, Manzil Zaheer, Andrew McCallum
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization
. EMNLP, 2022. [pdf]
Dhruv Agarwal, Rico Angell, Nicholas Monath, Andrew McCallum.
Entity Linking via Explicit Mention-Mention Coreference Modeling
. NAACL, 2022. [pdf]
Rico Angell, Nicholas Monath, Nishant Yadav, Andrew McCallum.
Interactive Correlation Clustering with Existential Cluster Constraints.
ICML, 2022. [pdf]
Siddhartha Mishra, Nicholas Monath, Michael Boratko, Ariel Kobren, Andrew McCallum.
An evaluative measure of clustering methods incorporating hyperparameter sensitivity.
AAAI, 2022. [pdf]
Archan Ray, Nicholas Monath, Andrew McCallum, Cameron Musco.
Sublinear Time Approximation of Text Similarity Matrices.
AAAI, 2022. [arxiv] [AAAI pdf]
Michael Boratko, Dongxu Zhang, Nicholas Monath, Luke Vilnis, Kenneth L Clarkson, Andrew McCallum.
Capacity and Bias of Learned Geometric Embeddings for Directed Graphs.
NeurIPS, 2021. [pdf]
Nicholas Monath, Avinava Dubey, Guru Guruganesh, Manzil Zaheer, Amr Ahmed, Andrew McCallum, Gokhan Mergen, Marc Najork, Mert Terzihan, Bryon Tjanaka, Yuan Wang, Yuchen Wu.
Scalable Hierarchical Agglomerative Clustering.
KDD. 2021. [arxiv] [kdd pdf] [scipy code] [cpp code, python bindings] [video]
Craig Greenberg*, Sebastian Macaluso*, Nicholas Monath*, Avinava Dubey, Patrick Flaherty, Manzil Zaheer, Amr Ahmed, Kyle Cranmer, Andrew McCallum.
Exact and Approximate Hierarchical Clustering with A*.
UAI. 2021. (* equal contribution). [arxiv] [video]
Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan and Andrew McCallum.
Scaling Within Document Coreference to Long Texts.
Findings of ACL, 2021. [pdf] [video]
Rico Angell, Nicholas Monath, Sunil Mohan, Nishant Yadav, Andrew McCallum.
Clustering-based Inference for Zero-Shot Biomedical Entity Linking.
NAACL, 2021. [arxiv]
Nicholas Monath, Manzil Zaheer, Avinava Dubey, Amr Ahmed, Andrew McCallum.
DAG-Structured Clustering by Nearest Neighbors
. AISTATS, 2021. [paper] [code] [video]
Craig S. Greenberg*, Sebastian Macaluso*, Nicholas Monath, Ji-Ah Lee, Patrick Flaherty, Kyle Cranmer, Andrew McGregor, Andrew McCallum.
Data Structures & Algorithms for Exact Inference in Hierarchical Clustering
. AISTATS, 2021. [arxiv] [code]
Rajarshi Das, Ameya Godbole, Nicholas Monath, Manzil Zaheer, Andrew McCallum.
Probabilistic Case-based Reasoning in Knowledge Bases
. Findings of EMNLP 2020 [arxiv] [code]
Dung Thai, Zhiyang Xu, Nicholas Monath, Boris Veytsman, Andrew McCallum.
Using BibTeX to Automatically Generate Labeled Data for Citation Field Extraction.
AKBC. 2020 [pdf]
Derek Tam, Nicholas Monath, Ari Kobren, Andrew McCallum.
Predicting Institution Hierarchies with Set-based Models
AKBC. 2020. [pdf]
Derek Tam, Nicholas Monath, Ari Kobren, Aaron Traylor, Rajarshi Das, Andrew McCallum.
Optimal Transport-based Alignment of Learned Character Representations for String Similarity.
ACL. 2019. [arxiv] [pdf] [code + data]
Nicholas Monath*, Ari Kobren*, Akshay Krishnamurthy, Michael Glass, Andrew McCallum.
Scalable Hierarchical Clustering via Tree Grafting.
KDD. 2019 (Oral presentation) (* Equal Contribution). [pdf] [arxiv] [code]
Nicholas Monath, Manzil Zaheer, Daniel Silva, Andrew McCallum, Amr Ahmed.
Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space.
KDD. 2019. [pdf] [code]
Nishant Yadav, Ari Kobren, Nicholas Monath, Andrew McCallum.
Supervised Hierarchical Clustering with Exponential Linkage.
ICML. 2019. [arxiv] [pdf] [code]
Ari Kobren, Nicholas Monath, Andrew McCallum.
Integrating User Feedback under Identity Uncertainty in Knowledge Base Construction.
AKBC, 2019. [pdf]
Craig Greenberg, Nicholas Monath, Ari Kobren, Patrick Flaherty, Andrew McGregor, Andrew McCallum.
Compact Representation of Uncertainty In Clustering.
NeurIPS 2018. [pdf]
Bo Xiao, Nicholas Monath, Shankar Ananthakrishnan, Abishek Ravi.
Play Duration based User-Entity Affinity Modeling in Spoken Dialog System
Interspeech 2018. [pdf]
Ari Kobren*, Nicholas Monath*, Akshay Krishnamurthy, and Andrew McCallum.
A Hierarchical Algorithm for Extreme Clustering
. KDD. 2017. (* equal contribution). (Oral Presenation). [pdf] [code] [talk] [promo video]

PhD Thesis

Incremental Non-Greedy Clustering at Scale.

[official university pdf]

Workshop & Other Publications

Nishant Yadav, Nicholas Monath, Rico Angell, Andrew McCallum.
Event & Entity Coreference using Trees to Encode Uncertainty in Joint Decisions.
Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC) at EMNLP 2021.
Sunil Mohan, Rico Angell, Nicholas Monath, Andrew McCallum. Low Resource Recognition and Linking of Biomedical Concepts from a Large Ontology. BCB. 2021. [arxiv]
Ethan Shen, Maria Brbic, Nicholas Monath, Jiaqi Zhai, Manzil Zaheer, Jure Leskovec.
Model-Agnostic Graph Regularization for Few-Shot Learning
. NeurIPS Workshop on Meta-Learning, 2020. [arxiv]
Nicholas Monath *, Ari Kobren*, Akshay Krishnamurthy, Andrew McCallum.
Gradient-based Hierarchical Clustering
. NIPS Workshop on Discrete Structures in Machine Learning. 2017. (Oral Presentation). [pdf]
Ari Kobren, Nicholas Monath, Andrew McCallum.
Entity-centric Attribute Feedback for Interactive Knowledge Bases.
NIPS Workshop on Automated Knowledge Base Construction. 2017. [pdf]
Aaron Traylor *, Nicholas Monath *, Rajarshi Das, Andrew McCallum.
Learning String Alignments for Entity Aliases
. NIPS Workshop on Automated Knowledge Base Construction. 2017. [pdf] [code]
Haw-Shiuan Chang, Abdurrahman Munir, Ao Liu, Johnny Tian-Zheng Wei, Aaron Traylor, Ajay Nagesh, Nicholas Monath, Patrick Verga, Emma Strubell, and Andrew McCallum.
Extracting Multilingual Relations under Limited Resources: TAC 2016 Cold-Start KB construction and Slot-Filling using Compositional Universal Schema.
NIST TAC KBP Workshop 2016. Notebook version [pdf]
Benjamin Roth, Nicholas Monath, David Belanger, Emma Strubell, Patrick Verga and Andrew McCallum
Building Knowledge Bases with Universal Schema: Cold Start and Slot-Filling Approaches
TAC KBP 2015 Workshop [pdf]
Nicholas Monath and Andrew McCallum.
Discriminative Hierarchical Coreference for Inventor Disambiguation
. PatentsView Inventor Disambiguation Technical Workshop. September 2015 [slides] [code]
Mykel J. Kochenderfer and Nicholas Monath.
Data Compression of Optimal Value Functions for Markov Decision Processes
Data Compression Conference. Snowbird Utah 2013.

Awards

1st place.

Inventor Disambiguation Challenge.

PatentsView Inventor Disambiguation Technical Workshop. September 2015. [link] [slides] [code]. Our inventor name disambiguation system was integrated into USPTO PatentsView website.

Education

Graduate:
University of Massachusetts Amherst.
Started MS (only) Fall 2013
Entered PhD Fall 2015

Undergraduate:
Brandeis University. BS 2013. Computer Science and Mathematics.
Graduated Summa Cum Laude.
Phi Beta Kappa.
Highest Honors for Undergraduate Thesis. Michtom Prize for Academic Excellence in Computer Science.

Misc

KDCOVID - April 2020 - We have released KDCOVID. KDCOVID retrieves papers by measuring similarity between queries and sentences in the full text of papers in CORD19 corpus. KDCOVID highlights entities linked to knowledge-bases and drug-gene-disease associations. Developed by Manzil Zaheer, Nicholas Monath, Shehzaad Dhuliawala, Taamannae Taabassum, Rajarshi Das, Bhuwan Dhingra, and Andrew McCallum. [www.kdcovid.nl] [Kaggle submission] [code on github]

Sets & Partitions - December 2019 - I co-organized The First Workshop on Sets and Partitions, was held as a part of the NeurIPS 2019 conference, with Manzil Zaheer, Ari Kobren, Junier Oliva, Barnabás Póczos, Ruslan Salakhutdinov, Andrew McCallum. The workshop was focused on models for tasks with set-based inputs/outputs as well as models of partitions and novel clustering methodology. [Workshop Site]

Talks

Scalable Hierarchical Clustering with Tree Grafting KDD 2019. [slides]

Optimal Transport-based Alignment of Learned Character Representations for String Similarity ACL 2019. [slides]

A Hierarchical Algorithm for Extreme Clustering KDD 2017. [poster] [slides from UMass Data Science Symposium]

Discriminative Hierarchical Coreference for Inventor Disambiguation PatentsView Inventor Disambiguation Technical Workshop. 2015. [slides]. (Our inventor name disambiguation system received 1st place and is a part of the USPTO PatentsView website).