Reinforcement learning methods of adaptive sampling for Seabed identification

Level set estimation (LSE)––finding contours which divide a domain into sets based on the value of a function––is a problem important to accurately modeling the acoustic properties of the sea floor for acoustic sensing, where we estimate similarities between seabed acoustic profiles at various locations. The vast scales involved with oceanic acoustic sampling using autonomous underwater vehicles can make naive or uniform approaches to spatial sampling locations expensive or infeasible. Modeling the value of the sampled function as a posterior Gaussian process (GP) of the sample values enables an efficient model that informs adaptive sampling policies. Previous approaches to LSE adaptive sampling have focused on deterministic algorithms with pre-programmed heuristics. As an alternative, we apply a reinforcement learning (RL) agent to the problem, which provides greater flexibility and customizability throughout the entire sampling trajectory. Instead of using the posterior GP to determine subsequent sample locations directly, we use it as an observational input for the RL agent. We test our method on synthetic binary fields generated as realizations of a separate GP and compare it to previously published deterministic algorithms, in terms of distance traveled, number of samples, and LSE accuracy.