Coordinating Multi-Agent Learners
Abstract: Multi-agent reinforcement learning (MARL) provides an attractive, scalable, and approximate approach for agents to learn coordination policies and adapt their behavior to the dynamics of the uncertain and evolving environment. However, for most large-scale applications involving hundreds of agents, current MARL techniques are inadequate. MARL may converge slowly, converge to inferior equilibria, or even diverge in realistic settings. There are no known distributed approaches that guarantee convergence without either very constraining assumptions about the learning environment and the knowledge at each agent or intractable amounts of computation and communication. These assumptions do not hold in most realistic applications. In this lecture, I will overview my group’s work over the last five years in melding multi-agent coordination technology with more complex single agent reinforcement learning for scaling MARL to large agent networks. This discussion will include the use of non-local multi-level supervisory control to coordinate and guide the agents’ learning process, the use of approximate DCOP algorithms for peer-to-peer learning coordination, the use of conflict resolution detection to dynamically expand the policy space of an agent so as to incorporate additional non-local information, and increasing the sophistication of local agent learning through policy prediction and multiple learning contexts. This is joint work with Dr. Chongjie Zhang, Professor Sherief Abdallah, Professor Anita Raja, Dr. Shanjun Cheng and Bruno Castro da Silva.
Bio: Victor Lesser received the Ph.D. degree in computer science from Stanford University, Stanford, CA, 1973. He is an Emeritus Distinguished Professor of Computer Science and Director of the Multi-Agent Systems Laboratory at the University of Massachusetts. His major research focus is on the control and organization of complex AI systems. He has pioneered work in the development of the blackboard architecture and its control structure, approximate processing for use in control and real-time AI, and a wide variety of techniques for the coordination of and negotiation among multiple agents. He was the system architect for first fully developed blackboard architecture (HEARSAY-II), when he was a research computer scientist at CMU from 1972 thru 1976, and is considered one of the founders of the Multi-Agent field starting with his early work in 1978. He has also made contributions in the areas of machine learning, signal understanding, diagnostics, plan recognition, and computer-supported cooperative work. He has worked in application areas such as sensor networks for vehicle tracking and weather monitoring, speech and sound understanding, information gathering on the internet, peer-to-peer information retrieval, intelligent user interfaces, distributed task allocation and scheduling, and virtual agent enterprises.
Professor Lesser's research accomplishments have been recognized by many major awards over the years. He received the prestigious IJCAI-09 Award for Research Excellence. He is also a Founding Fellow of AAAI and an IEEE Fellow. He was General Chair of the first international conference on Multi-Agent Systems (ICMAS) in 1995, and Founding President of the International Foundation of Autonomous Agents and Multi-Agent Systems (IFAAMAS). In 2007, to honor his contributions to the field of multi-agent systems, IFAAMAS established the “Victor Lesser Distinguished Dissertation Award.” He also received a Special Recognition Award for his foundational research in generalized coordination technologies from the Information Processing Technology Office at DARPA.