Quantifying the Sim2Real Gap: Model-Based Verification and Validation in Autonomous Ground Systems

Quantifying the Sim2Real gap is crucial for validating autonomous ground systems, enabling robust algorithm testing in simulations before real-world deployment, thereby reducing costs and time. This study introduces the Vinnicombe (-gap) metric as a quantitative tool for assessing this gap. To achieve this, a non-holonomic skid-steer differential drive robot was used. The -gap metric compares two dynamical control systems and returns a value between 0 and 1, where 0 indicates identical systems and 1 indicates significantly different systems. A linear time-invariant dynamic model, optimized through a genetic algorithm, was employed to ensure accurate representation of system behavior across varying conditions. Unlike task-specific metrics focused on localized errors, the -gap metric provides a holistic assessment by capturing system-wide differences. The -gap metric quantified significant differences with a maximum of 0. 64 between real-world and simulated trials highlighting discrepancies in vehicle-environment interactions. Terrain-induced changes in real-world comparisons were quantified with values up to 0. 27, reflecting increased compliance and friction on rubber-like surfaces versus concrete. Internal system changes were also identified with -gap values between 0. 25 and 0. 32, demonstrating sensitivity to changes in vehicle dynamics. These findings highlight the -gap metric's utility in enhancing simulation fidelity and reducing reliance on resource-intensive real-world testing.