Andrew G. Barto

Professor Emeritus
Retired Co-Director Autonomous Learning Laboratory

College of Information and Computer Sciences
University of Massachusetts Amherst
barto [at] cs [dot] umass [dot] edu



Note to Potential Grad-Student, Internship, and Post-Doc Applicants

Since I retired in 2012 I am no longer taking on new students or interns. I encourage potential applicants to look for other opportunities in the College of Information and Computer Sciences by looking here .



Publications

    More information on the 2nd edition

Ph.D. students

 

Research Interests

My research centers on learning in machines and animals. I worked on developing learning algorithms that are useful for engineering applications but that also make contact with learning as studied by psychologists and neuroscientists. Although I make no claims to being either a psychologist or a neuroscientist, I have spent a lot of time interacting with scientists in those fields and reading their books and papers. I think it is important that new developments should be integrated as closely as possible with the state-of-the art in as many of the relevant fields as possible. It is also important to understand how new developments relate to what others have done in the past. We all occasionally rediscover the wheel, but we should eventually recognize that we have!

In the case of reinforcement learning (RL)—whose main ideas go back a very long way—it has been immensely gratifying to participate in establishing new links between RL and methods from the theory of stochastic optimal control. Especially exciting are the connections between temporal difference (TD) algorithms and the brain's dopamine system. These are partly responsible for rekindling my interest in RL as an approach to building and understanding autonomous agents, rather than as a collection of methods for finding good solutions to engineering problems. The second edition of the RL book with Rich Sutton contains new chapters on RL from the perspectives of psychology and neuroscience.

An area of recent interest is about what psychologists call intrinsically motivated behavior, meaning behavior that is done for its own sake rather than as a step toward solving a specific problem of clear practical value. What we learn during intrinsically motivated behavior is essential for our development as competent autonomous agents able to efficiently solve a wide range of practical problems as they arise. The idea here is that the reward signals that an RL agent learns from do not always have to come from the agent's external environment. Reward signals can also come from within the agent itself to reward behavior that helps the agent acquire knowledge and skills that will be useful over its lifetime. This leads to fundamental questions about reward signals, both extrinsic and intrinsic. What makes a good reward signal? What kinds of intrinsic reward signals do our brains, and the brains of other animals, generate? How are these signals related to evolutionary fitness, and how have they evolved? Some of my recent papers with colleagues deal with these questions.

I am also interested in the many challenges that have to be faced as RL moves out into the real world. Most important is the challenge of making sure that RL makes positive contributions to our lives that outweigh any negative consequences. Visit the Autonomous Learning Laboratory page for some more details.


Top of page