CDP: Constrained Diffusion Policies with Mirror Diffusion Model for Safety-Assured Imitation Learning

Abstract

This paper presents a novel imitation learning framework, called constrained diffusion policy (CDP). The primary objective of CDP is to ensure that learned policies strictly adhere to safety constraints while imitating expert demonstrations. To achieve this, we define a polytopic constraint that represents the safe boundary for obstacle-free region. We introduce a novel mirror map and its inverse function to incorporate a generalized polytopic constraint manifold into the mirror diffusion model. By mapping sampled data onto a constrained manifold, the mirror diffusion model generates actions that satisfy safety constraints. This approach successfully addresses the safety issues commonly encountered in conventional imitation learning models. We apply the proposed framework to mobile navigation tasks in robotics, using the Isaac Gym simulator and the Unitree Go2 quadrupedal robot. Experimental results demonstrate that the proposed framework can successfully train policies that imitate expert behaviors while strictly maintaining safety constraints, thereby achieving safety-assured imitation learning.