A Regret-Informed Evolutionary Approach for Generating Adversarial Scenarios for Black-Box Off-Road Autonomy Systems
Ted Sender, Mark J. Brudnak, Reid Steiger, Ram Vasudevan, Bogdan I. Epureanu
Abstract
Developing autonomous vehicles (AVs) that operate in diverse and demanding environments is a difficult challenge. Two fundamental tools that can accelerate this process are testing an AV in diverse simulated environments and identifying core system weaknesses. While most efforts focus on improving these tools for on-road AVs, this paper focuses on an analogous set of tools for off-road AVs. A method called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Black-Box Adversarially Compounding Regret Through Evolution</i> (BACRE) is proposed for identifying adversarial scenarios using an evolutionary algorithm guided by a novel regret-based metric for general navigation tasks. A black-box approach is often preferable when system complexity can be diverse, like with off-road AVs, and when whole-system testing is required. A custom simulation platform is also provided to assist with the automated testing of AVs in diverse, unstructured environments. Numerical experiments demonstrate that BACRE's evolutionary process gradually increases scenario complexity to degrade vehicle performance (an effective and explainable process that comparable methods cannot achieve). Consequently, BACRE can streamline AV development by finding weaknesses at any development stage.
BibTeX
@inproceedings{ral2024_aregretinformede,
title = {A Regret-Informed Evolutionary Approach for Generating Adversarial Scenarios for Black-Box Off-Road Autonomy Systems},
author = {Ted Sender and Mark J. Brudnak and Reid Steiger and Ram Vasudevan and Bogdan I. Epureanu},
booktitle = {RA-L 2024},
year = {2024}
}