C-MPDM: Continuously-Parameterized Risk-Aware MPDM by Quickly Discovering Contextual Policies

Abstract

Risk-aware Multi-Policy Decision Making (MPDM)is a powerful framework for reliable navigation in a dynamic social environment where rather than evaluating individual trajectories, a “library” of policies (reactive controllers)is evaluated by anticipating potentially dangerous future outcomes using an on-line forward roll-out process. There is a core tension in Multi-Policy Decision Making (MPDM)systems - it is desirable to add more policies to the system for flexibility in finding good policies, however, this increases computational cost. As a result, MPDM was limited to small (perhaps 5-10)discrete policies - a significant performance bottleneck. In this paper, we radically enhance the expressivity of MPDM by allowing policies to have continuous-valued parameters, while simultaneously satisfying real-time constraints by quickly discovering promising policy parameters through a novel iterative gradient-based algorithm. Our evaluation includes results from extensive simulation and real-world experiments in semi-crowded environments.