Shape sensing by vision-based tactile sensor for dexterous handling of robot hands

In this paper, we propose a new method to estimate the shape and irregularity of objects by the vision-based tactile sensor, which is consist of a CCD camera, LED lights, transparent acrylic plate, and the touch pad which consists of the elastic membrane of silicon rubber and the translucent red water filling inside of the membrane. Intensities of red, green and blue bands of the lights are focused to estimate the shape of the object and the irregularity of the object surface. The LED light traveling in the translucent touch pad is scattered and absorbed by the particles of the red pigment which fill the inside of the touch pad. The depth of the pad is estimated by using the intensity of the light obtained from the red-green-blue (RGB) values in the image, in consideration of the scattering and reflection effects. The intensity of traveling light decays depending on the depth of the pad, the reflection coefficient to the membrane of the pad surface, the scattering coefficient of the pigment particle, and so on. The reflection and scattering coefficients that are considered very hard to obtain, are decoupled in the proposed formulation, and the intensity of traveling light is represented with the function of the geometrical parameters of the pad surface. In order to decrease the approximation error caused by unmodeled factors, the compensation is made by using the function of the shape of the membrane. The usefulness of the proposed method is confirmed through the experimental results. The shape sensing method is developed for dexterous handling by robot hands.