Reinforcement learning navigation strategy for Unmanned Vehicles with multimodal sensing fusion

To address the issues that single-sensor Unmanned Vehicles are prone to environmental constraints in navigation and obstacle avoidance tasks, and that reinforcement learning-based navigation and obstacle avoidance models suffer from low training efficiency and weak reward guidance, this paper constructs a multimodal perception framework centered on LiDAR and RGB cameras, introduces a curriculum learning strategy into the reinforcement learning training of unmanned vehicle navigation and obstacle avoidance models, and incorporates the core idea of the artificial potential field method into the reward function design. Experimental results demonstrate that the proposed method yields positive optimization for training stability; the multimodal approach remedies the defect of the frontal perception blind spot existing in 2D LiDAR, and the full sim-to-real workflow deployment of the model is accomplished, which verifies the effectiveness of the proposed method.