Microscopic Vision-guided Robotic Tissue Scanning

Advances in biophotonics have led to the development of the pCLE technology, which enables direct visualisation of tissue at microscopic scale during surgery. Preliminary research indicates that pCLE can effectively identify residual cancerous tissue, resulting in higher rates of tumor removal [1]. However, maintaining the pCLE probe within a working distance of micrometer range and orthogonal to the tissue surface, presents a significant challenge. Recently, a blur metric-based computer vision approach has been proposed to control the longitudinal distance between the pCLE probe and the tissue [2]. This approach, however, only approximates the probe’s location relative to the tissue surface rather than regressing their actual distance. In terms of pCLE orientation control, Zhang et al. [3] reconstructed the 3D map of the tissue surface and applied marker-based probe pose estimation to infer the probe’s pose with respect to the tissue surface during scanning. Nonetheless, this method’s effectiveness is limited when dealing with deforming tissue, and its accuracy can be affected by the surgical environment’s varying lighting conditions [4]. Most crucially, it cannot achieve the micrometer precision required for controlling the pCLE probe. To overcome these challenges and establish an au- tonomous robotic tissue scanning system, the Spatial- Frequency Feature Coupling Network (SFFC-Net) [5] and the Fast Fourier Vision Transformer (FF-ViT) [6] have been developed to provide visual feedback to the robotic scanning system. This approach enables the robotic scanning system to precisely approach the tissue surface and accurately adjust the pose of the probe. By integrating SFFC-Net and FF-ViT, we aim to improve the quality of pCLE data acquisition and facilitate more effective and accurate surgical procedures.