SVG-Mamba: An efficient local-to-global sparse voxel graph mamba for event stream recognition

Recent works based on Transformer or voxel architectures have demonstrated impressive performance in event stream recognition. However, due to their quadratic complexity or multiple dense operations, these methods potentially sacrifice efficiency or sparsity of events to achieve comparable results. To overcome this, we propose a sparse voxel graph Mamba (SVG-Mamba), which combines message-passing graph neural networks (GNNs) with state space models (SSMs) to efficiently model local-to-global spatiotemporal contextual relationships in variable-length event sequences, thereby mitigating this trade-off. Specifically, we introduce a vertex feature encoding (VFE) module that effectively compensates for the shortcomings of GNNs in temporal modeling while preserving event sparsity. To explore latent spatial semantics and motion cues, we design a lightweight local feature aggregator (LFA) consisting of point-wise spatial encoding (PWSE), an attention pooling unit (APU), and dilated residual connections (DRC). Benefiting from SSMs’ powerful global modeling ability at linear complexity, we construct hierarchical dynamic farthest vertex sampling (DFVS) and Mamba-based global attention to expand the local vertex receptive field into global spatiotemporal awareness. Extensive experimental results show that our proposed SVG-Mamba not only achieves higher accuracy but also exhibits significant advantages in terms of inference speed for event-based visual tasks such as object classification and action recognition. The source code is available at: https://github.com/hust-fstudy/SVG-Mamba .