Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports

Abstract

Recent advances in safe autonomy open new opportunities in assisting humans in safety-critical and time-sensitive tasks such as motorsports. However, existing safe control algorithms predominantly focus on fully automated settings and often undermine key requirements in human–AI shared control domains—maintaining human agency and respecting their decisions. Preserving human agency is crucial in human–AI collaboration to reduce the risks of “automation surprise”, where human operators may feel confused, uncomfortable, or even irritated by the AI’s assistance. In this work, we propose Human-Centered Safety Filter (HCSF), a principled shared autonomy framework that improves robustness in human decision-making while minimally affecting human agency. HCSF is based on a novel model-free safety formulation that allows the AI “assistant” to step in gradually, rather than at the last possible moment, to ensure that AI interventions feel smooth and are only as large as necessary. In contrast to conventional safety filters that prioritize safety without taking into account human decisions, HCSF minimally modifies human actions and actively promotes the transparency of AI assistance. We evaluate HCSF with Assetto Corsa, a high-fidelity car racing simulator, where human drivers are assisted by an AI assistant to race against an in-game opponent. Extensive human trials demonstrate that HCSF enhances safety without compromising the driver’s agency, leading to improved user confidence, comfort, and overall satisfaction.