Collaborative human-robot manipulation of highly deformable materials

Robotic manipulation of highly deformable materials is inherently challenging due to the need to maintain tension and the high dimensionality of the state of the material. Past work in this area mostly focuses on generating a detailed model for the material and its interaction with the robot, then using the model to construct a motion plan. In this paper, we take a different approach by using only sensor feedback to dictate the robot motion. We consider the collaborative manipulation of a deformable sheet between a person and a dual-armed robot (Baxter by Rethink Robotics). The robot is capable of contact sensing via joint torque sensors and is equipped with a head-mounted RGBd sensor. The robot senses contact force to maintain tension of the sheet, and in turn comply to the human motion. This is akin to handling a tablecloth with a partner but with one's eyes closed. To improve the response, we use the RGBd sensor to detect folds, and command the robot to move in an orthogonal direction to smooth them out. This is like handling cloth by looking at the cloth itself. Both controllers are able to follow human motion without excessive crimps in the sheet, but as expected, the hybrid controller combining force and vision outperforms the force controller alone in terms of tension force transient. The ability to quickly detect the state of the deformable material also enables more complex manipulation strategies in the future.