Perceptive Model Predictive Control for Continuous Mobile Manipulation

A mobile robot needs to be aware of its environment to interact with it safely. We propose a receding horizon control scheme for mobile manipulators that tracks task space reference trajectories. It uses visual information to avoid obstacles and haptic sensing to control interaction forces. Additional constraints for mechanical stability and joint limits are met. The proposed method is faster than state of the art sampling based planners, available as opensource and can be implemented on a broad class of robots. We validate the method both in simulation and through extensive hardware experiments with a multitude of mobile manipulation platforms. The resulting software package is released with this letter.