Tu Vu

Tu Vu

Manning College of Information and Computer Sciences
University of Massachusetts Amherst

Computer Science Building
140 Governors Drive, Amherst, MA 01003

Office: Room 266 - UMass NLP Lab
Email: tuvu@cs.umass.edu
CV // Google Scholar // Semantic Scholar // Twitter

About me

I am on the job market for academic and industry research positions starting in Fall 2023 or later. Here are my CV and research statement.

I am a Ph.D. candidate in the Manning College of Information and Computer Sciences (CICS) at University of Massachusetts Amherst (UMass Amherst), where I work with Mohit Iyyer (since Fall 2018) in the UMass Natural Language Processing group (UMass NLP). Currently, I also spend one day a week as a student researcher at Google Brain in Mountain View. Prior to joining UMass Amherst, I received my B.S. in Computer Science from Vietnam National University, Hanoi.

I have spent several semesters as a research intern/student researcher at Google Brain in Mountain View (Fall 2022, Winter 2021-22 & Spring 2022, Winter 2020-21 & Spring 2021), Google Brain in New York (Summer 2022), Google Research in Mountain View (Summer & Fall 2021, Summer & Fall 2020), and Microsoft Research in Montreal (Summer 2019).

Research

Along with my advisor, I work on problems that lie at the intersection of natural language processing (NLP) and deep learning. My research interests lie broadly in using transfer learning, unsupervised learning, and semi-supervised learning methods that exploit large volumes of unlabeled data or beneficial relationships among tasks to improve performance on tasks with limited or no labeled data. Currently, I am excited about prompt-based learning methods that use text prompts and/or "soft prompts" (a sequence of tunable tokens) to condition a frozen language model to perform different tasks.
Topics in this vein that I work on include:

I have contributed research code to multiple open-source libraries/repositories, including Google Research's repository, huggingface/transformers, fastai/fastai, allenai/allennlp. Check out a self-training algorithm I have recently added to huggingface/transformers here!

Recent News

Teaching

Check out my recent lecture on transfer learning with large-scale language models at VietAI here!

Teaching Assistant:

Fall 2021: Advanced Natural Language Processing (COMPSCI 685)
Fall 2020: Advanced Natural Language Processing (COMPSCI 685)

Fall 2019: Introduction to Natural Language Processing (COMPSCI 585)

Fall 2018: Introduction to Natural Language Processing (COMPSCI 585)

Spring 2018: Machine Learning (COMPSCI 589)

Service & Outreach

Program Committee/Reviewer: AAAI 2023; NeurIPS 2022; ARR 2022, 2021; ACL 2022, 2021, 2020, 2019; EMNLP 2022; 2021; NAACL-HLT 2022; 2021; COLING 2020; CoNLL 2019; INLG 2020, 2019
Co-reviewer/Secondary Reviewer: ICLR 2019, 2020; NAACL-HLT 2019

Preprints/Publications

For an up-to-date list of my papers, please see my Google Scholar profile.

Selected Publications


SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Cer
ACL 2022
One of the three projects highlighted in the Headlines of Google AI’s Natural Language Accelerated Newsletter, Q1, 2022
PDF Slides Poster


Exploring and Predicting Transferability across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
EMNLP 2020
Taught/discussed in several NLP courses, including COMP790-101 by Colin Raffel at UNC Chapel Hill, CS 685 by Mohit Iyyer at UMass Amherst, ECE 594 by Suma Bhat at UIUC
PDF Slides Colin Raffel's tweet


STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
Tu Vu, Minh-Thang Luong, Quoc V. Le, Grady Simon, Mohit Iyyer
EMNLP 2021
Press: UMass Amherst & Google Improve Few-Shot Learning on NLP Benchmarks via Task Augmentation and Self-Training // Machine Learning - Reddit
The most popular Arxiv paper the day it came out!
PDF Slides


Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, Noah Constant
EMNLP 2022
PDF Poster


The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, Adam Roberts
ICML 2023
Google Research's Blog: The Flan Collection: Advancing open source methods for instruction tuning
The current best “open-source” large language models for both few-shot prompting and fine-tuning // Hugging Face's models
PDF



All Recent Preprints/Publications


Flan-MoE: Scaling Instruction-Finetuned Language Models with Sparse Mixture of Experts
Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou
arXiv 2023
PDF

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, Adam Roberts
ICML 2023
PDF


Dialect-robust Evaluation of Generated Text
Jiao Sun, Thibault Sellam, Elizabeth Clark, Tu Vu, Timothy Dozat, Dan Garrette, Aditya Siddhant, Jacob Eisenstein, Sebastian Gehrmann
ACL 2023
PDF


Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, Noah Constant
EMNLP 2022
PDF Poster


Leveraging QA Datasets to Improve Generative Data Augmentation
Dheeraj Mekala, Tu Vu, Timo Schick, Jingbo Shang
EMNLP 2022
PDF


SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Cer
ACL 2022
PDF Slides Poster


STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
Tu Vu, Minh-Thang Luong, Quoc V. Le, Grady Simon, Mohit Iyyer
EMNLP 2021
PDF Slides


Exploring and Predicting Transferability across NLP Tasks.
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
EMNLP 2020
PDF Slides


Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification.
Tu Vu, Mohit Iyyer
ACL 2019
PDF Poster

Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment.
Tu Vu, Vered Shwartz
*SEM@NAACL-HLT 2018
PDF


Sentence Simplification with Memory-Augmented Neural Networks.
Tu Vu, Baotian Hu, Tsendsuren Munkhdalai, Hong Yu
NAACL-HLT 2018
PDF



Patents
* Original inventor

Frozen Model Adaptation through Soft Prompt Transfer
Tu Vu*, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Cer
U.S. Patent Application Serial No. 17/863,840

Task Augmentation and Self-training for Improved Few-shot Learning
Minh-Thang Luong, Tu Vu*, Quoc V. Le, Grady Simon
U.S. Patent Application Serial No. 17/826,690


Selected Awards, Honors, and Funding


Distant Past: