A Cooperative Routing Protocol Based on Q-Learning for Underwater Optical-Acoustic Hybrid Wireless Sensor Networks

Underwater wireless networking and communications methodologies, including the routing algorithms, have been broadly studied these years. This paper designs an optical-acoustic hybrid network modeling approach, taking the characteristics of both the underwater optical and acoustic waves into account. Underwater nodes are divided into two types, namely intra-cluster and cluster head nodes, in which energy consumption and time delay in underwater optical and acoustic links are both mathematically modeled and TDMA MAC protocol is reasonably applied in acoustic and optical links. Based on these preliminaries, a cooperative routing protocol is proposed using the Q-learning technique, where the forwarding actions are determined according to received rewards and experiential knowledge. To be specific, sender nodes will evaluate the effects of all possible forwarding actions on the network and routing performances before selecting the receiver nodes. It is shown by simulation experiment results that this routing protocol outperforms three relevant noted underwater routing algorithms in terms of network connectivity rate, lifetime, energy efficiency, energy consumption balance, packet-loss rate, and end-to-end delay.