MADR: MPC-Guided Adversarial Deepreach

Abstract

Hamilton-Jacobi Reachability offers a framework for generating safe value functions and policies in the face of adversarial disturbance, but is limited by the curse of dimensionality. Physics-informed deep learning is able to overcome this infeasibility, but itself suffers from slow and inaccurate convergence, primarily due to weak PDE gradients and the complexity of self-supervised learning. Recent works have demonstrated that enriching the self-supervision process with regular supervision (based on the nature of the optimal control problem) greatly accelerates convergence and solution quality; however, these have been limited to single-player problems and simple games. In this work, we introduce MADR: MPC-guided Adversarial DeepReach, a general framework to robustly approximate the two-player, zero-sum differential game value function. In doing so, MADR yields the corresponding optimal strategies for both players in zero-sum games as well as safe policies for worst-case robustness. We test MADR on a multitude of high-dimensional simulated and real robotic agents with varying dynamics and games, finding that our approach significantly outperforms state-of-the-art baselines in simulation and produces impressive results in hardware.