Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

In this paper we establish rigorous benchmarks for image classifier. Our first benchmark, ImageNet-C, standardizes and expands the robustness topic, while showing which classifiers are preferable in-critical applications. Then we propose a new dataset called ImageNet-P enables researchers to benchmark a classifier's robustness to common. Unlike recent robustness research, this benchmark evaluates on common corruptions and perturbations not worst-case adversarial. We find that there are negligible changes in relative corruption from AlexNet classifiers to ResNet classifiers. Afterward we ways to enhance corruption and perturbation robustness. We even find a bypassed adversarial defense provides substantial common perturbation. Together our benchmarks may aid future work toward networks that generalize.