DEOGFormer: a dual–branch edge–optimized global network for real–time semantic segmentation of omnidirectional mine roadway images

This is the authors' abstract. We don't add key points for this paper.

Accurate environmental perception is a prerequisite for the safe operation of driverless mine electric locomotives. Due to the limitations of traditional methods, most researchers have shifted towards vision-based rail perception. However, the unique value of omnidirectional mine roadway images (OMRI) in perception is often overlooked. Simultaneously, the specific environment of mine roadways imposes higher demands on the precise extraction of edge information and the real-time performance of algorithms. To address these challenges, this paper proposes DEOGFormer, a Dual-Branch Edge-Optimized Global Network for real-time semantic segmentation of OMRI. Recognizing the Transformer’s excellent performance in global information extraction but its shortcomings in capturing local detailed features, we designed an Edge Feature Guided Branch (EFGB) in the encoding stage. This branch complements the main branch by enhancing the capture of edge details and local information. Subsequently, we introduce a Cross-Channel Feature Adaptive Screening and Fusion Module (CCFASFM) to better fuse features from the two branches, enabling the model to focus on more salient regions and enhancing feature integration capabilities. To overcome the limitations of traditional additive or multiplicative feature fusion methods when handling multi-feature integration, we designed a Multi-Level Attention Fusion Decoder (MLAFD) in the decoding stage. MLAFD dynamically adjusts through an attention mechanism, achieving deep interaction among four types of features, thus ensuring that critical features receive higher weights during the fusion process. Furthermore, to support our research, we constructed an Omnidirectional Mine Roadway Semantic Segmentation Dataset (OMRSSD), comprising 4500 meticulously annotated images across 13 annotation categories. On this dataset, DEOGFormer delivers 86.86% mIoU and 91.1 FPS, while maintaining a lightweight footprint of merely 3.46 M parameters and 5.72 GFLOPs. To further verify the generalization capability of the proposed model, we perform additional experiments on the PASCAL VOC2012 and CamVid datasets, achieving highly satisfactory performance.