Camera–LiDAR Data Fusion for Enhanced Ship Situational Awareness in Maritime Environment

Reliable obstacle detection and classification are essential capabilities for the safe and efficient navigation of Marine Autonomous Surface Ships. This paper introduces a decision-level multi-sensor fusion framework to enhance situational awareness for autonomous vessels by integrating RGB camera and LiDAR data. Visual information is processed using a pre-trained, open-source object detection model. At the same time, LiDAR measurements are analysed with a clustering-based algorithm, followed by a lightweight Random Forest classifier for semantic labelling. To support practical deployment in real maritime environments, the proposed approach relies on readily available perception modules, avoiding the need for training on proprietary datasets and limiting dependence on extensive task-specific tuning. The fusion of these complementary sources is employed to confirm and characterise dynamic obstacles, whose positions derived from LiDAR are continuously tracked using a Global Nearest Neighbour algorithm supported by a Kalman filter. Each stage of the proposed processing chain is thoroughly described and experimentally validated using real-world data collected in a representative marine environment, demonstrating the approach’s effectiveness in improving perception performance by reducing false positives from noisy measurements and achieving 92% track number accuracy in a complex scenario.