%0 Journal Article
%T A Novel Approach to Locomotion Learning: Actor-Critic Architecture using Central Pattern Generators and Dynamic Motor Primitives
%A Cai Li
%A Robert Lowe
%A Tom Ziemke
%J Frontiers in Neurorobotics
%D 2014
%I Frontiers Media
%R 10.3389/fnbot.2014.00023
%X In this article, we propose an architecture of a bio-inspired controller that addresses the problem of learning different locomotion gaits for different robot morphologies. The modelling objective is split into two: baseline motion modelling and dynamics adaptation. Baseline motion modelling aims to achieve fundamental functions of a certain type of locomotion and dynamics adaptation provides a ``reshaping" function for adapting the baseline motion to desired motion. Based on this assumption, a three-layer architecture is developed using central pattern generators (CPGs, a bio-inspired locomotor center for the the baseline motion) and dynamic motor primitives (DMPs, a model with universal ``reshaping" functions). In this article, we use this architecture with the actor-critic algorithms for finding a good ``reshaping" function. In order to demonstrate the learning power of the actor-critic based architecture, we tested it on two experiments: 1) learning to crawl on a humanoid and, 2) learning to gallop on a puppy robot. Two types of actor-critic algorithms (policy search and policy gradient) are compared in order to evaluate the advantages and disadvantages of different actor-critic based learning algorithms for different morphologies. Finally, based on the analysis of the experimental results, a generic view/architecture for locomotion learning is discussed in the conclusion.
%K actor-critic
%K Central pattern generators (CPG)
%K reinforcement learning
%K Locomotion control
%K NAO robot
%U http://www.frontiersin.org/Journal/10.3389/fnbot.2014.00023/abstract