Dynamic Obstacle Avoidance Navigation for USVs Based on the Multihead Attention Mechanism and Distributional Reinforcement Learning

Autonomous navigation of unmanned surface vehicles (USVs) is challenged by environmental uncertainty and safety risks caused by eddies and unknown obstacles. Traditional deep reinforcement learning based on expected returns is prone to decision bias when confronted with environmental uncertainty, as it tends to overlook the distributional characteristics of returns. Distributional reinforcement learning provides richer statistical information for risk modeling by learning the distribution of returns, but it still lacks the ability to focus on key risk areas within environmental observations. To address this problem, a local path-planning method, referred to as the multihead attention mechanism implicit quantile network (MHA-IQN), is proposed based on the multihead attention mechanism and the distributional RL. The attention module fuses obstacle features from sensors such as LiDAR with the USV’s status information. It generates a normalized risk score to represent environmental risk by calculating the attention weights for each obstacle and performing a weighted aggregation. This risk score is then subjected to a parameterized monotonic mapping and smoothed by exponential moving average to obtain the instantaneous conditional value-at-risk (CVaR) confidence level, which is used for quantile sampling in the IQN, thereby achieving risk-sensitive value estimation and decision-making. Simulation results in environments with random Rankine vortices and circular obstacles show that the proposed method outperforms IQN with fixed CVaR thresholds, DQN, and classical reactive algorithms in safety, navigation time, and energy consumption. By dynamically adjusting the obstacle avoidance conservativeness level based on environmental risk, the method enables the USV to prioritize high-risk obstacles and improve overall navigation performance.