Multi-criteria prioritized reinforcement learning for autonomous underwater vehicle navigation in 3D multi-node environments

Autonomous underwater vehicles (AUVs) are increasingly used as mobile data collectors in underwater wireless sensor networks, where navigation must balance task completion, obstacle avoidance, communication performance, and energy efficiency. This study develops a Twin Delayed Deep Deterministic Policy Gradient framework with Multi-Criteria Prioritized Experience Replay (MC-PER-TD3). The replay priority combines normalized temporal-difference error with a goal-proximity heuristic; energy consumption is incorporated into the reward and data-collection evaluation rather than the replay priority. A soft potential-field reward and a time-window-based waypoint escape-transition mechanism are further used to guide continuous navigation. Across six training seeds and 60 deterministic-policy evaluation episodes per seed, MC-PER-TD3 achieved 100% mission success with zero observed collisions. Relative to uniform-replay TD3, the mean successful path length decreased from 13.188 ± 3.885 m to 10.985 ± 0.436 m and task time from 6.911 ± 1.511 s to 5.654 ± 0.257 s. A standard TD-error-only PER-TD3 baseline achieved 50% mission success, whereas the combined TD-error-plus-goal replay achieved consistent completion across all six training seeds. Additional zero-shot tests under observation noise, current perturbations, fixed and randomized obstacle layouts reveal condition-dependent robustness and clear failure boundaries. In UWSN evaluation, MC-PER-TD3 achieved the highest mean energy efficiency, while TD3 retained the highest throughput, indicating a throughput-energy trade-off rather than universal dominance.