Full-Array Markers Tracking Adapted to Large Contact Distortion in Visuotactile Sensing

Visuotactile sensors (VTS) achieve tactile sensing based on camera-captured images, where the problem of losing markers during tracking may lead to inferior sensor performance. This paper proposes a methodology adapted to large contact distortion in VTS sensing, including the geometric constrained marker pattern design, and the dynamic full-array markers tracking method. The tracking method couples the traditional sparse optical flow (SOF) and the lost marker relocation algorithm, thus it maintains high accuracy in the non-contact region, while the complementary markers are patched in the contact region ensuring the integrity of tracking markers when localized large distortion occurs. Based on the numerical benchmark, the proposed method is compared with the conventional SOF method (two marker patterns with array or checkerboard markers). The results show that, in the case of large deformation, the leakage rate of the tracking markers with the proposed method (0%) is significantly lower compared to that of SOF methods (>10%). Furthermore, a compact binocular VTS is prototyped, where the press-release tests with multiple types of objects are performed, and the contact sensing performance is analyzed. Full-array markers can be tracked dynamically with root mean square error of tracking at the sub-pixel level (0.26 pixels). In terms of dynamic force sensing, the proposed method is capable of tracking sudden changes in contact force by relocating the lost markers, whereas the traditional methods are usually ineffective.