Development of Automatic Target Detection System for 11,000M-Class Underwater Vehicles

An underwater vehicle with a tether cable is called a remotely operated vehicle (ROV), and an operator on the mother ship can observe video images and collect samples in real time. However, when operating in very deep sea, the tether cable becomes extremely long, and its handling is technically difficult. Against this background, the authors are currently developing a vehicle that can operate at great depths without relying on tether cables. In this case, the vehicle requires a higher degree of autonomy since the operator cannot remotely control the vehicle. As a technology necessary for autonomous sample collection, we have developed a system that automatically detects and tracks targets from camera images. Machine learning methods are widely used as a means of detecting target objects from camera images. We adopted Detection Transformer (DETR) from among multiple methods as a target detection model. We then created an underwater model by training it with videos of deep-sea creatures and dummy targets recorded by our organization. Since camera images in the underwater environment are different from those on land, we applied data augmentation to add lens distortion and blur during training. We also compared multiple methods for tracking models for detected objects and implemented the Byte-Track algorithm. The developed system consists of an underwater camera and an image processing device, each housed in a pressure-resistant container designed to be used at great depths. The underwater camera uses a stereo camera system so that it can detect and track the target and measure its position at the same time. The image processing device was sized to fit in a pressure-resistant container, and hardware capable of image processing using a graphics processing unit (GPU) was selected, and the processing performance was confirmed. After basic confirmation of the developed system through tank tests, it was installed on an 11,000m-class lander owned by our organization and a demonstration test was conducted at a triple junction (depth of 9,200m). Since the possibility of encountering living things at this depth is low, a dummy target was also attached to the lander to confirm detection. As a result, the system successfully detected and tracked targets in real time. On the other hand, detection performance sometimes deteriorated in certain situations such as hiding behind objects. In order to improve the issues identified this time, the authors plan to conduct additional learning using newly acquired video data. Through continued testing and evaluation, we plan to install it on 11,000m class vehicles in the future.