RVFNet: 4D mmWave Radar-Vision Fusion Multi-Task Panoptic Perception Network for Unmanned Surface Vehicles

This is the authors' abstract. We don't add key points for this paper.

4D mmWave radar-vision fusion is a low-cost and promising solution for Unmanned Surface Vehicles navigation. However, existing methods mainly focus on single tasks and suffer from insufficient radar feature extraction and inefficient multi-modal fusion. This paper proposes RVFNet, a 4D mmWave radar-visual fusion multi-task perception network. It designs the Radar Feature Enhancement Module (RFEM) and alternately stacks RFEM with conventional convolution blocks to construct the Radar Feature Extraction Network (RFENet), which solves the problem of insufficient extraction of sparse and irregular radar point cloud features and suppresses water-surface clutter; it constructs the Multi-modal Feature Adaptive Fusion Module (MFAFM) with content-guided attention. MFAFM generates dynamic modal weights and adaptively balances visual and radar information to alleviate missed detection and false detection in harsh scenarios. Compared with mainstream methods, experimental results on the WaterScenes dataset show that RVFNet requires only 3.3 GFLOPs while achieving an mAP50 of 79.0%, an mAP50--95 of 46.7%, an mIoU of 84.0% for object segmentation, and an mIoU of 98.8% for drivable-area segmentation, demonstrating superior overall perception performance. Cross-scenario experiments on the FloW-RI water-surface dataset and the nuScenes v1.0-trainval autonomous-driving dataset further verify the strong generalization capability of RVFNet, demonstrating its potential for deployment on embedded devices.