Sim-on-Wheels: Physical World in the Loop Simulation for Self-Driving
Yuan Shen, Bhargav Chandaka, Zhi-Hao Lin, Albert J. Zhai, Hang Cui, David A. Forsyth, Shenlong Wang
Abstract
We present Sim-on-Wheels, a safe, realistic, and vehicle-in-loop framework to test autonomous vehicles' performance in the real world under safety-critical scenarios. Sim-on-wheels runs on a self-driving vehicle operating in the physical world. It creates virtual traffic participants with risky behaviors and seamlessly inserts the virtual events into images perceived from the physical world in real-time. The manipulated images are fed into autonomy, allowing the self-driving vehicle to react to such virtual events. The full pipeline runs on the actual vehicle and interacts with the physical world, but the safety-critical events it sees are virtual. Sim-on-Wheels is safe, interactive, realistic, and easy to use. The experiments demonstrate the potential of Sim-on-Wheels to facilitate the process of testing autonomous driving in challenging real-world scenes with high fidelity and low risk. Additional results and open-sourced code are available on our project page here: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sim-on-wheels.github.io/</uri> .
BibTeX
@inproceedings{ral2023_simonwheelsphysi,
title = {Sim-on-Wheels: Physical World in the Loop Simulation for Self-Driving},
author = {Yuan Shen and Bhargav Chandaka and Zhi-Hao Lin and Albert J. Zhai and Hang Cui and David A. Forsyth and Shenlong Wang},
booktitle = {RA-L 2023},
year = {2023}
}