Vision-based Tip Force Estimation on a Soft Continuum Robot

Soft continuum robots, fabricated from elastomeric materials, offer unparalleled flexibility and adaptability, making them ideal for applications such as minimally invasive surgery and inspections in constrained environments. With the miniaturization of imaging technologies and the development of novel control algorithms, these devices provide exceptional opportunities to visualize the internal structures of the human body. However, there are still challenges in accurately estimating external forces applied to these systems using current technologies. Adding additional sensors is challenging without compromising the softness of the device. This work presents a visual deformation-based force sensing framework for soft continuum robots. The core idea behind this work is that point loads lead to unique deformation profiles in an actuated soft-bodied robot. We introduce a Convolutional Neural Network-based tip force estimation method that utilizes arbitrarily placed camera images and actuation inputs to predict applied tip forces. Experimental validation was performed using the STIFF-FLOP robot, a pneumatically actuated soft robot developed for minimally invasive surgery. Our vision-based force estimation model demonstrated a sensing precision of 0.05 N in the XY plane during testing, with data collection and training taking only 70 minutes.