Occlusion-Aware 6D Pose Estimation with Visual Observation Guided Diffusion Model

Abstract

Category-level 6D pose estimation in cluttered and occluded environments is a challenging task. Most existing methods rely on deterministic point-based correspondences to estimate target poses, which cannot consider the uncertainty for occluded objects, and thus result in inferior performance. In this paper, we propose a diffusion model guided by occlusion-aware observations to adaptively refine the object poses in occluded and cluttered scenes. Specifically, we first extract various 2D and 3D features from an RGB-D image to construct the conditions of diffusion model. In the reverse diffusion process, the model is guided by implicit correspondences, perception distance, and occlusion relationships to refine the noisy pose sampled from a standard Gaussian distribution. With several denoising steps, our method can produce accurate results that are consistent with image observations in occluded scenarios. The experimental results show that the proposed method can outperform baseline methods in major metrics in occlusion scenarios. Furthermore, our approach can also be applied in robotic grasping and manipulation tasks through grasping experiments in a cluttered enviroment on a physical UR5 robot.