Diffusion Models are Good Unsupervised Class-agnostic Shape Part Segmentators

Abstract

Shape part segmentation is a critical task in computer graphics and robotics. However, traditional supervised methods rely heavily on large amounts of labeled data, which poses significant challenges in many real-world scenarios where such data is often scarce or difficult to obtain. To address this issue, we propose an unsupervised, class-agnostic part segmentation method called Point Diffusion Segmentation (PDS). Our research demonstrates that unconditional point cloud diffusion models can capture abstract object concepts within their sub-attention layers. By extracting preliminary point cloud features from these attention maps, PDS generates efficacious segmentation results. This method fully leverages unlabeled data and proves to be highly applicable in various downstream tasks, including zero-shot part segmentation. Without resorting to any labeled data, PDS improves the zero-shot part segmentation performance of PointClipV2 by 3.1% on the ShapeNet Part dataset, setting a new state-of-the-art baseline and demonstrating significant potential of PDS.