UMass Machine Learning and Friends Lunch | Main / Unraveling Unsupervised Feature Learning

Successful applications of Machine Learning typically require us to expend significant effort engineering new features and representations before applying a learning algorithm. For the most challenging applications in AI, like Computer Vision, the search for good high-level representations is vast and ongoing. Recently, many researchers have sought to create algorithms that can learn features from data automatically. In particular, a growing variety of Unsupervised Feature Learning (UFL) algorithms are now able to learn good representations from unlabeled datasets. Yet despite steadily improving benchmark scores, it is unclear what exactly makes some algorithms perform well and others perform poorly.

In this talk, we'll start with an overview of a standard UFL pipeline and investigate the various factors that can affect the performance of feature learning. Through several sets of experiments, a surprising picture emerges: we will find that many schemes succeed or fail as a result of factors completely unrelated to their particular learning methods. In fact, by focusing solely on these factors and using very simple learning algorithms, it is often possible to outperform all prior art on widely-used benchmarks. Though perhaps briefly discouraging, our experiments will also suggest some other paths to explore. In this direction, we will introduce a new algorithm that learns the connectivity structure of multi-layered networks of features. Our results not only include improved benchmark scores, but also tantalizing evidence that attempts to learn deep hierarchies of features entirely without supervision may yet succeed.