Synthetic Datasets for Autonomous Driving: A Survey

Autonomous driving techniques have been flourishing in recent years while thirsting for huge amounts of high-quality data. However, it is difficult for real-world datasets to keep up with the pace of changing requirements due to their expensive and time-consuming experimental and labeling costs. Therefore, more and more researchers are turning to synthetic datasets to easily generate rich and changeable data as an effective complement to the real world and to improve the performance of algorithms. In this paper, we summarize the evolution of synthetic dataset generation methods and review the work to date in synthetic datasets related to single and multi-task categories for the autonomous driving perception study. We also discuss the role that synthetic datasets play in the evaluation, gap test, and positive effect of autonomous driving-related algorithm testing, especially on trustworthiness and safety aspects, and provide examples of evaluation experiments. Finally, we discuss the limitations and future directions of synthetic datasets. To the best of our knowledge, this is the first survey focusing on the application of synthetic datasets in autonomous driving. This survey also raises awareness of the problems of real-world deployment of autonomous driving technology and provides researchers with a possible solution.