Machine Learning and Friends Lunch
Hierarchical Policy Gradient Algorithms
Mohammad Ghavamzadeh
UMass
Abstract
Hierarchical reinforcement learning is a general framework which attempts to accelerate policy learning in large domains. On the other hand, policy gradient reinforcement learning methods have received recent attention as a means to solve problems with continuous state and/or action spaces. However, they suffer from slow convergence. In this work, we combine these two approaches and propose a family of hierarchical policy gradient algorithms for problems with continuous state and/or action spaces. We also introduce a class of hierarchical hybrid algorithms, in which a group of subtasks, usually at the higher-levels of the hierarchy, are formulated as value function-based reinfrocement learning problems and the others as policy gradient reinforcement learning problems.