Characterizing open-set ATR methods to detect out of distribution data for a maritime object classifier in space-based electro optical imagery

Reliable computer vision object classification is important for security applications that make high stakes decisions based on automated algorithms. In real world scenarios, it is often impractical to meet the implicit assumption that all relevant, labelled data may be attained prior to training. To avoid performance degradation, a recently developed open-set detection framework is applied to the classification of ships from clutter in satellite, Electro-Optical (EO) imagery and is shown to reliably identify data that is out of distribution from training data. A Binary Classifier (BC) and Category-aware Binary Classifier (CBC) model were compared to OpenMax and found to provide improvements in identifying unknown imagery. This enables an operator to know whether to believe classification results from a deep learning-based algorithm.