A simulator-based approach for testing and assessing human supervised autonomous ship navigation

Abstract This paper proposes and demonstrates a simulator-based approach for testing and assessing human operators’ ability and performance in supervising autonomous ships. In the autonomy concept studied here, it is assumed that the autonomous navigation system is capable of detecting and notifying the human operator prior to entering a challenging situation. The system will attempt to resolve the situation with a proposed evasive maneuver, but the system may occasionally make errors or select sub-optimal solutions. When the human operator is notified about a challenging situation, the operator should closely assess the situation, and intervene if (and only if) necessary. The proposed approach allows us to test and quantify the human operators’ abilities and performances in supervising an autonomously navigated ship. The approach is demonstrated on 56 simulator experiments, involving seven different navigators that perform eight different traffic scenarios. The scenarios are all based on real traffic data collected from a Norwegian ferry-crossing. The demonstration shows that the candidates are successful in supervising the autonomous ship in low-complexity traffic scenarios where it is easy for the operator to interpret the system’s decisions. The operators’ ability to intervene when and only when needed does, however, decline in more complex scenarios, and their performance is highly dependent on the traffic scenario and vary significantly between the candidates.