Learning Controllers for Reactive and Proactive Behaviors in Human–Robot Collaboration

Designed to safely share the same workspace as humans and assist them in a variety of tasks, the new collaborative robots are targeting manufacturing and service applications that once were considered unattainable. The large diversity of tasks to carry out, the unstructured environments and the close interaction with humans call for collaborative robots to seamlessly adapt their behaviors so as to cooperate with the users successfully under different and possibly new situations (characterized, for example, by positions of objects/landmarks in the environment, or by the user pose). This paper investigates how controllers capable of reactive and proactive behaviors in collaborative tasks can be learned from demonstrations. The proposed approach exploits the temporal coherence and dynamic characteristics of the task observed during the training phase to build a probabilistic model that enables the robot to both react to the user actions and lead the task when needed. The method is an extension of the Hidden Semi-Markov Model where the duration probability distribution is adapted according to the interaction with the user. This Adaptive Duration Hidden Semi-Markov Model (ADHSMM) is used to retrieve a sequence of states governing a trajectory optimization that provides the reference and gain matrices to the robot controller. A proof-of-concept evaluation is first carried out in a pouring task. The proposed framework is then tested in a collaborative task using a 7 DOF backdrivable manipulator.