Co-Optimizing Reconfigurable Environments and Policies for Decentralized Multi-Agent Navigation

Abstract

This work views the multi-agent system and its surrounding environment as a co-evolving system. The goal is to take agent actions and environment configurations as decision variables, and optimize both in a coordinated manner. Towards this end, we consider the problem of decentralized multi-agent navigation in reconfigurable environments. By introducing two sub-objectives of multi-agent navigation and environment optimization, we propose an agent-environment co-optimization problem and develop a coordinated algorithm that alternates between sub-objectives to search for an optimal synthesis of agent actions and obstacle configurations; ultimately, improving navigation performance. Due to the challenge of modeling the relation between agents, environment and performance, we formulate a model-free learning mechanism within the coordinated framework. A formal convergence analysis shows our coordinated algorithm tracks the local minimum trajectory of an associated time-varying non-convex optimization problem. Experiments evaluate the benefits of co-optimization and interestingly, indicate optimized environments offer structural guidance that is key to de-conflicting agents.