NIIRS, machine learning, and confidence: issues and opportunities for automating analysis NIIRS

Image exploitation has evolved with a growing reliance on machine learning (ML). With increased reliance on ML comes questions about performance, reliability, and trust. When an imagery analyst performs analysis without the assistance of an ML model, the confidence is dependent on the quality of the underlying imagery and the expertise of the analyst. The National Imagery Interpretation Rating Scale (NIIRS) maps task complexity to image quality based on human cognition. Thus, NIIRS is an accepted standard for the information potential of the image for human analysis. Empirical analysis indicates that NIIRS is, at best, a partial indicator of expected performance for ML. This paper explores several factors that can affect the quality of ML-based results: • The image quality as assessed in the ML context, • The scene complexity of the image, • The ML architecture, and • The relationship between the training imagery and the mission imagery. This paper will explore each of these factors and discuss their importance. We conclude with a set of recommendations for future research.