Quantifying the Sim2Real Gap: Model-Based Verification and Validation in Autonomous Ground Systems

Abstract

Quantifying the Sim2Real gap is crucial for validating autonomous ground systems, enabling robust algorithm testing in simulations before real-world deployment, thereby reducing costs and time. This study introduces the Vinnicombe (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\nu$</tex-math></inline-formula>-gap) metric as a quantitative tool for assessing this gap. To achieve this, a non-holonomic skid-steer differential drive robot was used. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\nu$</tex-math></inline-formula>-gap metric compares two dynamical control systems and returns a value between 0 and 1, where 0 indicates identical systems and 1 indicates significantly different systems. A linear time-invariant dynamic model, optimized through a genetic algorithm, was employed to ensure accurate representation of system behavior across varying conditions. Unlike task-specific metrics focused on localized errors, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\nu$</tex-math></inline-formula>-gap metric provides a holistic assessment by capturing system-wide differences. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\nu$</tex-math></inline-formula>-gap metric quantified significant differences with a maximum of 0.64 between real-world and simulated trials highlighting discrepancies in vehicle-environment interactions. Terrain-induced changes in real-world comparisons were quantified with values up to 0.27, reflecting increased compliance and friction on rubber-like surfaces versus concrete. Internal system changes were also identified with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\nu$</tex-math></inline-formula>-gap values between 0.25 and 0.32, demonstrating sensitivity to changes in vehicle dynamics. These findings highlight the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\nu$</tex-math></inline-formula>-gap metric's utility in enhancing simulation fidelity and reducing reliance on resource-intensive real-world testing.