Vision-based reinforcement learning for humanoid behavior generation with rhythmic walking parameters

This paper presents a method for generating vision-based humanoid behaviors by reinforcement learning with rhythmic walking parameters. The walking is stabilized by a rhythmic motion controller such as CPG or neural oscillator. The learning process consists of two stages: the first one is building an action space with two parameters (a forward step length and a turning angle) that inhibits combinations that are not feasible. The second is reinforcement learning with the constructed action space and the state space consisting of visual features and posture parameters to find feasible actions. The method is applied to a situation of the RoboCupSoccer humanoid league [H. Kitano and M. Asada, Advanced Robotics, 2000], that is, to approach the ball and to shoot it into the goal. Instructions by human are given to start up the learning process and the rest is completely self-learning in real situations.