Single-View 3D Object Perception Based on Vessel Generative Adversarial Network for Autonomous Ships

3D object perception in maritime is a promising means to enhance the ability of the autonomous navigation. However, 3D automatic reconstruction of dynamic targets remains a challenging task in the 3D object perception systems. In this paper, we present an innovative GAN-based deep learning neural network for 3D ship reconstruction. The method is able to reconstruct 3D ship models from single view images or videos using shape prior knowledge and a vessel generative adversarial network (Vessel-GAN), which can effectively extract the ship shape features from the single-view images or videos. Furthermore, a 3D feature network optimization model is proposed for loop optimization of 3D ship shape perception. In order to validate the effectiveness of the proposed method, a high resolution single ship dataset (HSSD) was created, and the performance of the proposed method was evaluated against the existing methods, The experimental results show that the Vessel Segment Network (VSNet) can improve the accuracy of the HSSD dataset by 2.5% in the single ship segmentation tasks. In addition, Vessel-GAN was demonstrated to be able to generate multi-view ship images with an impressive FID value of 5.82. Moreover, our model can improve the accuracy of 3D ship modeling by 8.9% on the ShapeNet dataset. This innovative approach promises to enable 3D perception of the marine environment for autonomous ship.