Simulation fidelity analysis using deep neural networks

Machine learning algorithms require datasets that are both massive and varied to train and generalize effectively. However, preparing real-world semantically labeled datasets is a very time-consuming and cumbersome task. Also, training with low volume datasets can lead to compromised performance and poor generalization of such algorithms. This algorithm performance and generalization gap due to limited quantities of real-world data could be decreased with the help of synthetic datasets that are generated with the consideration of real-world features. In this work, a combination of synthetic and real-world datasets is used to demonstrate and assess the performance of simulated-to-real-world transfer learning algorithms where the training is done in synthetic and testing in real-world datasets. The performance is further evaluated with a mixture of real and synthetic datasets. Two simulators are used in this work to generate synthetic images. The first was the Mississippi State University Autonomous Vehicle Simulator (MAVS), a high-fidelity physics-based simulator for Autonomous Ground Vehicle (AGV) in off-road terrain. The MAVS has been used to study machine learning in a variety of applications using both camera and lidar data. In addition to MAVS, the Unreal Engine version 4 (UE4) was used to generate images. Finally, images with a variety of synthetic scene fidelities and real-world images were considered for training the neural network to evaluate the effectiveness of low-fidelity synthetic data and the network performed very well with excellent confidence scores for object detection.