Machine Learning and Friends Lunch

home
past talks
resources
conferences

Object Based Abstraction Using MDP Homomorphisms

Abstract

One of the major problems facing "real" learning agents in the world is the complex sensory input they receive. This complexity results in an enormous the state space, in which the same state, strictly speaking, may never be encountered twice. One solution which could enable agents to plan and act in the face of this overwhelming input is to use the fact that there is structure in the environment to create a simplified (but accurate) model of the world. The particular structure we can use in this case is the fact that the world consists of many separate, yet interlocking pieces -- objects.

Consider the following sequence of events: a bat hits a ball, which flies through the air and shatters a window. We would like to model each independent piece -- bat, ball, window -- yet at the same time capture their interdependencies.

In order to do so, we create multiple object-based homomorphic images of a large uderlying Markov Decision Process without ever constructing the entire MDP, or even visiting its entire state space. This talk will discuss these types of object homomorphism and methods for finding them.