Exploiting the Relationship within the Unlabelled Samples by Set Matching for Generalized Category Discovery

Abstract

Generalized Category Discovery (GCD) is an open-world problem in semi-supervised learning, where the model leverages labelled data from known classes to discover both known and unknown classes in unlabelled data. Contrastive learning is key to this process, helping generate discriminative features that separate different categories. However, while labelled data benefits from direct supervision, unlabelled data relies solely on learning through instance discrimination, which focuses on consistency between different views of the same sample. This approach often neglects relationships between samples, limiting performance in GCD tasks. To improve this, we propose a set-matching perspective and formulate the problem as an optimal transport task to learn the coupling between batches of unlabelled data and a candidate pool. Our extensive experiments demonstrate that this approach better preserves the semantic structure of unlabelled data, leading to consistent improvements in baseline models and achieving state-of-the-art performance on various benchmarks.