Improving instance segmentation model for teeth in oral endoscope images based on lightweighting YOLO-TOE

The detection and accurate segmentation of teeth in endoscopic images are crucial for image navigation and robotic surgery in dental practice. Most multi-step methods result in slow speed, cumbersome models, and fragmented algorithms. This paper proposes a lightweight YOLO-TOE model to address the above problem. Specifically, the bottleneck in C2f is replaced by the MBConv Block containing the attention mechanism, and the SE-squeezed excitation network module in it is replaced with effective SE, resulting in C2f-EMBC, which highlights the key feature information with fewer parameters, suppresses the useless features, and avoids information loss. BiFPN is used instead of the original PAN as the neck network to enhance the feature fusion effect. A lossless structured pruning strategy is introduced to compress the computation and parameter count of the model layer by layer, and migrate the knowledge of the pruned network through a large-scale teacher network to enhance the prediction accuracy of the pruned model, which reduces the redundant computation and memory access and thus extracts the spatial features more efficiently. The experimental results show that YOLO-TOE achieves the MAP 50 of 95.7%; MAP 50 and MAP 50-95 have improved by 0.95% and 1.15% respectively compared to the baseline model. In addition, the Params and complexity are reduced by 67.44% and 32.5%, and the weight model is only 2.4 MB in size. This model outperforms other algorithms and satisfies the speed, precision, and cost-effectiveness requirements for tooth segmentation. Moreover, it is more suitable for deployment in medical systems.