Task-oriented synthetic-to-real image translation for data-efficient learning

Training state-of-the-art image classifiers and object detectors remains an extremely data-intensive process to this day. This is because inherently data-hungry, deep supervised networks are the traditional framework of choice. The significant data needs in turn impose strict requirements on the data acquisition, curation, and labelling stages that typically precede the learning process. This poses a particularly significant challenge for military and defense applications where the availability of high-quality labeled data is often limited. What is needed are methods that can effectively learn from sparse amounts of labeled, real-world data. In this paper, we propose a novel framework that incorporates a synthetic data generator into a supervised learning pipeline in order to enable end-to-end co-optimization of the discriminability and realism of the synthetic data, as well as the performance of the supervised engine. We demonstrate, via extensive empirical validation on image classification and object detection tasks, that the proposed framework is capable of learning from a small fraction of the real-world data required to train traditional, standalone supervised engines, while matching or even outperforming its off-the-shelf counterparts.