Visuo-haptic interaction of soft tissues for MR-based laparoscopic training

Abstract Visual and haptic interaction of soft tissue objects is crucial for the development of virtual reality (VR)-based training systems for laparoscopic surgery. However, most existing methods reported thus far suffer from limited computational efficiency and struggle to model the dynamic haptic interaction behavior between soft tissues and surgical instruments, which significantly restricts the sense of presence and practicality of the training system. Hence, this work proposes a visuo-haptic simulation approach for soft tissue deformation and its dynamic haptic interaction behavior. With the approach, an Extended Position-Based Dynamics (XPBD)-based soft tissue simulation method is proposed by integrating time-varying stiffness and damping coefficients, which specifically enables the realistic simulation of the dynamic damping oscillatory behavior of soft tissue during interaction with surgical instruments. Next, an interaction force model between soft tissue and surgical instruments is established, and a mixed reality (MR) training system for hepatobiliary separation surgery is developed by integrating haptic devices. The system allows users to operate a haptic device as if handling real surgical instruments, enabling realistic contact and interaction with virtual soft tissue. With real-time, high-fidelity deformation simulation and haptic feedback, users can efficiently perform common surgical maneuvers such as pulling, retracting, and sweeping soft tissues. The experimental results show that the proposed approach improves the simulation accuracy of dynamic haptic interaction behavior of soft tissue compared with actual soft tissue, which significantly enhances the training efficiency and quality of laparoscopic surgery.