A Decentralized Partially Observable Markov Decision Process for Dynamic Obstacle Avoidance and Complete Area Coverage using Multiple Reconfigurable Robots

Abstract

Achieving complete area coverage in robotics is an essential aspect for applications such as cleaning and patrolling. While multi-agent frameworks have been implemented to address the challenge of complete coverage, the area coverage performances are hindered by physical constraints and dynamic obstacles that cause inaccessibility to certain areas of the environment. Reconfigurable robots have been adopted to mitigate this issue as the independent alteration of the morphologies during deployments enables overcoming tight spaces to access obstructed areas. Hence, this paper proposes a Multi-Agent Reinforcement Learning (MARL) framework leveraging the Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to enable a team of reconfigurable robots to achieve complete coverage under the presence of dynamic obstacles. The framework is modelled to allow the robots to coordinate and plan their motions effectively while using shape adaptability to access narrow spaces while avoiding dynamic obstacles. Experimental results demonstrated the framework’s ability to be generalised even when scaled up to a different number of agents across larger environments.