Humans exhibit a strong predisposition to imagine — to mentally transcend time, place, and circumstance — from an early age. Cave paintings and sculptures found in Europe from tens of thousands of years ago show that our ancestors were capable of depicting or carving impossible objects. Imagination is found in every mythology in the world, from the Greeks to the Hindus. Imagination is prized in the arts, literature, music, poetry, as well as science and engineering.

In this tutorial, we introduce a new challenge for artificial intelligence: how to build imagination machines? Much of the recent excitement in AI is based on advances in data science, which is broadly the study of methods that convert samples into probability distributions. Data science is the study of “What is”: like statistics, it studies the summarization of historical data. We introduce a new field of study in AI called imagination science that, in contrast, is the study of “What if”? and “Why?”. Imagination science extends data science to answer a much broader range of questions, ranging from interventions to impossible counterfactual situations. We show that a number of converging lines of research in AI can be seen as attempts to build imagination machines, ranging from recent work on generative adversarial networks to cognitive architectures that combine observation, intervention and counterfactual reasoning.

We summarize novel research ideas, including new ways of modeling sequential decision making using counterfactual imagination models, as well as extensions of GANs using ideas from network economics. We show that research on imagination forms a nice synergy to ongoing work on high fidelity complex simulation engines, representing today’s forerunners to The Matrix, a “What if” eventual successor to today’s “What is” search engines. This tutorial provides a detailed discussion of key challenges in automating imagination, discuss connections between ongoing research and imagination, and outline why automation of imagination provides a powerful launching pad for transforming AI.

Click the "Course Description" link above to download the slides.