Stereo-vision based diver pose estimation using LSTM recurrent neural networks for AUV navigation guidance

Within the EU FP7 project “Cognitive autonomous diving buddy (CADDY)”, work has been made to assist and monitor divers through Autonomous Underwater Vehicles (AUVs) during their long underwater expeditions. To achieve this goal, one milestone is to give the AUV the capability to track the diver's whereabouts at all times. Inertial sensors are mounted on the diver's body to transmit acoustically his relative position and orientation to the AUV using an Ultra-Short Baseline (USBL) system. However, the acoustic system is prone to give erroneous data depending on the surrounding geography, modems alignment and sensors calibration, plus its low transmission rate does not allow real time performance. To overcome these drawbacks and complement the acoustic set-up, this paper presents a general framework to detect and estimate the diver's body pose based on generated point clouds from a stereo camera. Our method is based on a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) that captures the temporal relations between global point cloud descriptors as they change with the diver movements. Since the analysis is made on time sequences rather than on one-shot visual information, the framework is robust against the distortions and poor quality typical in underwater while still providing real time performance. We focus on the description of the image-processing and LSTM-RNN pipeline, as well on its validation with a dataset created during several field trial experiments.