A reinforcement learning approach for path discovery in MANETs with path caching strategy

In this paper, we enhance an existing path discovery scheme called the ticket-based probing (TBP) which supports QoS routing in mobile ad hoc networks (MANETs) to increase its accumulated reward. The scenario of QoS routing in MANETs with the presence of network information uncertainty is considered and modelled as a partially observable Markov decision process (POMDP). The proposed scheme integrates the original TBP scheme with a reinforcement learning method for POMDPs, called the on-policy first-visit Monte Carlo (ONMC) method, and a suitable path caching strategy. Simulation results shows that the inclusion of patch caching with the ONMC method can indeed achieve message overhead reduction with marginal difference in the path search ability and additional computational and storage requirements.