Research on Actor-Critic Reinforcement Learning in RoboCup

Actor-critic method combines the fast convergence of value-based (critic) and directivity on search of policy gradient (actor). It is suitable for solving the problems with large state space. In this paper, the actor-critic method with tile-coding linear function approximation is analysed and applied to a RoboCup simulation subtask named "Soccer Keepaway". The experiments on Soccer Keepaway show that the policy learned by actor-critic method is better than policies from value-based Sarsa(lambda) and benchmarks