Interlinked Visual Tracking and Robotic Manipulation of Articulated Objects

Robotic manipulation tasks require the knowledge on the configuration of the object in use. Since most objects are generally not equipped with any sensor, an estimator is required. Furthermore, if an object is articulated, i.e., includes passive joints, the estimation process has to reconstruct the pose of the object floating base and its joints variables, concurrently with the manipulation control. We address the estimation problem with an online virtual visual servoing paradigm written as a quadratic program. Our estimator is integrated in closed-loop with the manipulation control governing the robot, which is also a quadratic program. Tracking and manipulation experiments, using a humanoid, show the effectiveness of our approach.