Methods for non-intrusive out-of-distribution images detection

Selecting representative data is a key factor in improving the performance of machine learning algorithms. In this paper we focus on out-of-distribution (OoD) methods evaluation, which can be integrated into ML project lifecycle in a nonintrusive way, without changing a model architecture. Considered methods are applicable to image classification datasets analysis. In addition to commonly used AUROC metric, we evaluate the number of out-of-distribution samples misclassified with high confidence. Case studies were conducted on benchmark and production datasets. As a result, we provide practical guidance for data evaluation and recommendations on which method to use to detect different types of OoD images.