UMass Machine Learning and Friends Lunch | Main / Lazy Structured Prediction

Abstract:

Machine learning practitioners often face a fundamental trade-off between expressiveness and computation time: on average, more accurate, expressive models tend to be more computationally intensive both at training and test time. This tradeoff is acutely present in the setting of structured prediction, where the joint prediction of multiple output variables creates two inter-related bottlenecks: inference and feature computation time. In this talk, I will discuss my efforts to address the latter bottleneck in my PhD research. The key question I will discuss is, "Can we learn to compute features only as needed for maximum efficiency?"

I propose an architecture that uses a rich feedback loop between extraction and prediction. The run-time control policy is learned using efficient value-function approximation, which adaptively determines the value of information of features at the level of individual variables for each input. We demonstrate significant speedups over state-of-the-art methods on two challenging datasets. For articulated pose estimation in video, we achieve a more accurate state-of-the-art model that is simultaneously 4X faster while using only a small fraction of possible features, with similar results on an OCR task. Finally, I'll discuss my recent efforts to apply these techniques to natural language processing.

Bio:

David Weiss joined Google as a Research Scientist in 2014 after graduating with a Ph.D from the University of Pennsylvania under the supervision of Ben Taskar, receiving the Rubinoff Award for best CIS dissertation for his work on improving the efficiency and accuracy of structured prediction methods. Previously, he has worked on a variety of research problems in machine learning, statistics, neuroscience, recommender systems, computer vision, and natural language processing, finishing in 2nd place in the Netflix Prize competition and the CHALEARN One-Shot Learning Challenge. A recipient of an NSF Graduate Research Fellowship, he graduated in 2007 from Princeton University with a degree in Computer Science and a certificate in Neuroscience.