UMass Machine Learning and Friends Lunch | Main / Learning From Many Experiments Using Regularized Instrumental Variable Methods Applications To Peer Effects In Online Networks

Abstract

The widespread adoption of randomized experiments (i.e. A/B tests) in the Internet industry means that there are often numerous well-powered experiments on a given product. Individual experiments are often simple "bake-off" evaluations of a new intervention: They allow us to estimate effects of that particular intervention on outcomes of interest, but they are often not informative about the mechanisms for these effects or what other inventions might do. We consider what else we can learn from a large set of experiments. In particular, we use many experiments to learn about the effects of the various endogenous variables (or mechanisms) via which the experiments affect outcomes. This involves treating the experiments as instrumental variables, and so this setting is similar to, but somewhat different from, "many instrument" settings in econometrics and biostatistics. Motivated by the distribution of experiment first-stage effects, we present and evaluate sparsity-inducing regularization methods and cross-validation for instrumental variables. Our applications are to estimating peer effects in online social networks mediated by ranking systems.

Joint work with Alex Peysakhovich (Facebook) and including additional joint work with Eytan Bakshy (Facebook) and René Kizilcec (Stanford).

Bio

Dean Eckles is a social scientist, statistician, and faculty at MIT. Dean is the KDD Career Development Professor in Communications and Technology, an assistant professor in the MIT Sloan School of Management, and affiliated faculty at the MIT Institute for Data, Systems & Society. He was previously a member of the Core Data Science team at Facebook. He studies how interactive technologies affect human behavior by mediating, amplifying, and directing social influence — and statistical methods to study these processes. Dean’s empirical work uses large field experiments and observational studies. His research appears in the Proceedings of the National Academy of Sciences and other peer-reviewed journals and proceedings in statistics, computer science, and marketing. Dean holds degrees from Stanford University in philosophy (BA), cognitive science (BS, MS), statistics (MS), and communication (PhD).