A metamorphic testing technique for object detection networks

Context: Object detection networks are widely used in safety-critical domains such as autonomous driving and security monitoring. Failures in these systems can lead to severe loss of life and property. Therefore, rigorous testing with diverse and high-quality test cases is essential to ensure the quality and reliability of object detection systems. Objective: Traditional testing approaches face the oracle problem and difficulties in generating sufficient test data. To address these challenges, this paper employs metamorphic testing to automatically generate test cases for object detection networks. Method: We propose OBMeta, a metamorphic testing technique that rapidly constructs diverse, high-quality test cases for object detection networks. OBMeta decomposes each input image into foreground objects and the background by leveraging annotation information. Five transformation types, including brightness adjustment, noise addition, blurring, contrast adjustment, and foreground object insertion, are applied separately to the foreground, background, and entire image to construct metamorphic relations. Derived test cases are generated in batches and filtered via image similarity to preserve realism. Results: Experiments on seven object detection networks, including six single-stage YOLO and SSD variants and one two-stage Cascade R-CNN R50-FPN model, demonstrate that OBMeta efficiently generates test cases with higher similarity to original images, and yields a larger number of adversarial examples. Furthermore, retraining the networks with the detected adversarial examples improves model accuracy. Conclusion: OBMeta efficiently generates test cases that are both high-quality and error-detectable. It provides a practical and automated solution for testing object detection systems, thereby contributing to the improvement of their quality.