This paper proposes a Q-Learning-Based Distributed Energy-Efficient Routing (QDER) protocol for underwater acoustic sensor networks (UASNs). The routing problem is formulated as a Markov Decision Process (MDP) and a distributed Q-learning approach is proposed. Each sensor node is treated as an agent that independently selects its next-hop node based on a Q-table. The rewards function is designed that jointly considers node residual energy and depth information, enabling each node to learn an effective routing policy through distributed decision-making. Unlike centralized routing approaches that rely on extensive global information exchange, the proposed scheme allows nodes to make local decisions, thereby reducing communication overhead and energy consumption while maintaining efficient routing paths. In addition, link quality is designed in the reward to account for channel conditions, which improves the robustness of the routing strategy under noisy underwater acoustic environments. Simulation results demonstrate that the QDER achieves better system performance compared with Depth-Based Routing (DBR) and Deep Q-Network-Based Intelligent Routing (DQIR). Considering channel attenuation and noise, the proposed method with the link quality metric achieves improved network lifetime and energy efficiency. It also shows good robustness and adaptability under different signal-to-noise ratio (SNR) conditions.
Geng et al. (Thu,) studied this question.