Key result
MM-OR dataset and MM2SG model establish a new benchmark for holistic operating room understanding.
Why the study?
Current operating room datasets fall short in scale and realism and do not capture the multimodal nature of scenes, limiting progress in modeling for surgical assistance, situational awareness, and patient safety.
The introduction of the MM-OR dataset and MM2SG model establishes a new benchmark for multimodal scene analysis and holistic understanding of complex operating room environments.
Supports AI tool development for OR analysis; leaves open prospective clinical validation before adoption.
Operating rooms (ORs) are complex, high-stakes environments requiring precise understanding of interactions among medical staff, tools, and equipment for enhancing surgical assistance, situational awareness, and patient safety. Current datasets fall short in scale, realism and do not capture the multimodal nature of OR scenes, limiting progress in OR modeling. To this end, we introduce MM-OR, a realistic and large-scale multimodal spatiotemporal OR dataset, and the first dataset to enable multimodal scene graph generation. MM-OR captures comprehensive OR scenes containing RGB-D data, detail views, audio, speech transcripts, robotic logs, and tracking data and is annotated with panoptic segmentations, semantic scene graphs, and downstream task labels. Further, we propose MM2SG, the first multimodal large vision-language model for scene graph generation, and through extensive experiments, demonstrate its ability to effectively leverage multimodal inputs. Together, MM-OR and MM2SG establish a new benchmark for holistic OR understanding, and open the path towards multimodal scene analysis in complex, high-stakes environments. Our code, and data is available at https://github.com/egeozsoy/MM-OR.
No takes yet. Share an insight, caveat, or question.
Özsoy et al. (2025) studied Operating room environments. MM-OR dataset and MM2SG model was evaluated on Semantic scene graph generation and multimodal scene analysis. The MM-OR dataset and MM2SG model establish a new benchmark for holistic operating room understanding and multimodal scene graph generation.
Synapse has enriched 4 closely related papers on similar clinical questions. Consider them for comparative context: