Autonomous Systems Design, Testing, and Deployment: Lessons Learned from The Deployment of an Autonomous Shuttle Bus

Advances in technology have enabled the deployment of unprecedented levels of automation that verge on completely autonomous systems such as unmanned passenger and cargo vehicles, and air traffic control supported by integrated communications, navigation and surveillance (ICNS) systems.One application of the new technologies is in autonomous shuttle buses. This paper describes an analysis of a collision between an autonomous shuttle bus and delivery tractor-trailer on an urban street in Las Vegas. The analysis provides lessons learned for the design, testing, and fielding of future autonomous systems. First, the analysis demonstrates the difficulty in designing for all the "corner-cases" for safe fielding of an autonomous system. Second, the analysis shows the difficulty in demonstrating safety compliance to a target level of safety for systems developed using machine learning that cannot be tested using traditional testing methods (e.g. code-inspection or forms of input-output testing. Third, the analysis identifies the need for the explicit, intentional design, not an afterthought, of the task of the "safety driver." Solutions to these three issues are discussed.