Streamlined Detection of Key Diagnostic and Surgical Events in Medical Videos using CNN Architectures

Aim/Introduction: Detection of key diagnostic and surgical events in medical videos using CNN architectures. Background: In the medical sector, where viewing long procedural videos like cataract surgeries and endoscopies is prevalent, video summarization is essential for condensing vast amounts of video data. Unsupervised learning techniques are becoming more practical for this task because annotated data is expensive to collect. Objective: To propose different CNN architectures for extracting multi-layer spatial features, enhanced by multi-scale pyramid pooling, for robust and resolution-aware medical video summarization. Methods: The base, intermediate, and complex level features extracted from CNN-based encoders, namely MobileNetV3, GoogLeNet, DenseNet-161, ResNet-50, and EfficientNet-B7, are processed through a Spatial Pyramid Pooling (SPP) layer for finding multi-scale features. Furthermore, it is combined with a Bi-LSTM-based decoder for temporal modelling. Finally, an unsupervised reward-based pipeline is used for generating video summaries. Results and Discussion: The experimental results reveal that among the architectures tested, intermediate- level features from DenseNet-161 and EfficientNet-B7 with certain SPP bin settings perform exceptionally well on critical surgical procedures. However, MobileNetV3 with complex-level features performs well for surgical preparation and initial tool settings, while GoogLeNet and ResNet-50 deliver outstanding results on general-purpose datasets where high-level semantics are crucial. These configurations highlight a patent-worthy innovation in adapting CNNs and SPP techniques for video summarization. Conclusion: This paper presents an unsupervised encoder-decoder framework to capture multi-scale spatio-temporal features for generating robust video summaries that include key diagnostic and surgical events in the medical domain.