Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

The ability to detect objects regardless of image distortions or weather is crucial for real-world applications of deep learning like driving. We here provide an easy-to-use benchmark to assess how detection models perform when image quality degrades. The three benchmark datasets, termed Pascal-C, Coco-C and Cityscapes-C, contain large variety of image corruptions. We show that a range of standard object models suffer a severe performance loss on corrupted images (down to30--60\\% of the original performance). However, a simple data augmentation---stylizing the training images---leads to a substantial increase in across corruption type, severity and dataset. We envision our benchmark to track future progress towards building robust object models. Benchmark, code and data are publicly available.