Optimizing Spatial State Representation in Reinforcement Learning for Coverage Path Planning in UAV Search Missions

To enhance path planning efficiency in unmanned aerial vehicle (UAV) search missions in complex environments, this paper proposes a coverage path planning (CPP) algorithm for a UAV that integrates the deep Q-network (DQN) with the A* algorithm (DQN-A*). In the proposed DQN-A* algorithm, a dual-driven reward mechanism is established, comprising a probability-weighted reward and a step-dependent reward, steering the UAV toward high-probability regions. Furthermore, to handle previously unknown obstacles in real time, the algorithm employs a multi-stage obstacle-identification strategy, enabling the UAV to improve coverage of traversable cells by dynamically adjusting its local path when newly detected obstacles are encountered. A theoretical analysis derives a principled recommended range for the UAV positional identifier based on statistical feature analysis; this range is then validated through extensive simulations. Additionally, Hamiltonian path pre-training is introduced to accelerate convergence. Comparative simulations demonstrate that the proposed DQN-A* algorithm achieves higher area-coverage and target-detection probabilities than benchmark algorithms in environments with unknown obstacles, offering valuable insights for positional encoding in deep reinforcement learning (DRL)-based robotic coverage problems.