T-3MS fusion: gradient-aware transformer-guided middle-fusion for real-time 3D object detection on affordable sensor suite

High-precision 3D perception in autonomous driving remains constrained by dependence on expensive LiDAR sensors and computationally intensive models. These prohibitive requirements effectively limit resource-constrained platforms from accessing advanced perception capabilities, hindering the widespread adoption of autonomous technology. We present T-3MS Fusion, a transformer-based middle-fusion framework that achieves state-of-the-art 3D object detection performance using only a Velodyne VLP-32C LiDAR and a consumer-grade 360° camera, eliminating the need for test-time augmentation while maintaining computational efficiency. In contrast to early-fusion strategies that weaken spatial fidelity and late-fusion methods that lose geometric consistency, T-3MS employs a transformer-based middle-deep fusion architecture. This design leverages hierarchical gated residual transformers and adaptive cross-modal reactivation to preserve LiDAR geometry and camera semantics while enabling effective multi-scale feature integration. Sparse bird’s-eye-view processing and quantization-aware training enable real-time inference on embedded platforms. Validation on the nuScenes benchmark confirms strong performance with 74.9% NDS and 72.8% mAP, while evaluation on a self-collected semi-urban dataset acquired with low-cost and accessible hardware demonstrates resilience under occlusion, adverse illumination, and sparse point-cloud conditions, where even high-resolution LiDAR systems often experience significant performance degradation. These results establish T-3MS Fusion as an effective approach for jointly advancing accuracy, efficiency, and affordability in next-generation autonomous driving perception systems.