Runway safety program evaluation with uncertainties of benefits and costs

Risk and safety programs need to be monitored and evaluated through real-world experience and performance. Several perspectives on the program might be useful across organizational units, stakeholders, and subject-matter experts. Quantifications of risks, benefits, and costs of the programs as well as the associated parameter uncertainties will vary by perspective. The literature has quantified the benefits of a safety program in single stakeholder/expert perspectives. This paper describes a layering of perspectives of program effectiveness and characterizes the variation of estimated program efficacy across the several perspectives. Within each perspective, the uncertainties of model selection and the estimation of cost-benefit analysis parameters are addressed via interval numbers. The approach is demonstrated for safety trainings that are aimed to reduce runway incursions for 80 airports across a region, an issue of highest priority for the US National Transportation Safety Board. The several perspectives of the demonstration are the pilot/driver, the airport owner/operator, and the safety regulator. The results suggest for each of the perspectives which of the airports should receive the trainings. The paper will be of interest for the performance evaluation of safety programs with uncertainties of program benefits and costs and multiple agencies, users, customers, and other stakeholders/experts.