Soft Robotic Finger Embedded with Visual Sensor for Bending Perception

SUMMARY The various vision-based tactile sensors have been developed for robotic perception in recent years. In this paper, the novel soft robotic finger embedded with the visual sensor is proposed for perception. It consists of a colored soft inner chamber, an outer structure, and an endoscope camera. The bending perception algorithm based on image preprocessing and deep learning is proposed. The boundary of color regions and the position of marker dots are extracted from the inner chamber image and label image, respectively. Then the convolutional neural network with multi-task learning is trained to obtain bending states of the finger. Finally, the experiments are implemented to verify the effectiveness of the proposed method.