Exploiting previous experience to constrain robot sensorimotor learning

A truly autonomous robot should be able to generalize known actions to new situations and to autonomously refine its knowledge base. In this paper we present a three stage approach to the problem of expanding and refining the database of sensorimotor knowledge. The first stage is based on the generalization of previously trained movements associated with a specific task, which results in a first approximation of a suitable control policy in a new situation. The second stage applies learning on the manifold defined by the previously acquired training data, which results in a learning problem of reduced dimensionality. The final tuning of the desired control policy is accomplished by learning in the full state space, where the dimensionality of the problem is much higher. The assumption is that the first two steps already provide a good initial estimate for the optimal control policy so that this final step only locally refines the parameters learned in the first two steps. This significantly reduces the number of test trials needed by standard reinforcement learning techniques. The proposed approach was evaluated in simulation as well as on the real robot in a ball throwing experiment.