CLUSTSEG: Clustering for Universal Segmentation
James Chenhao Liang, Tianfei Zhou, Dongfang Liu, Wenguan Wang
Abstract
We present CLUSTSEG, a general, transformer-based framework that tackles different image segmentation tasks ($i.e.,$ superpixel, semantic, instance, and panoptic) through a unified, neural clustering scheme. Regarding queries as cluster centers, CLUSTSEG is innovative in two aspects: 1) cluster centers are initialized in heterogeneous ways so as to pointedly address task-specific demands ($e.g.,$ instance- or category-level distinctiveness), yet without modifying the architecture; and 2) pixel-cluster assignment, formalized in a cross-attention fashion, is alternated with cluster center update, yet without learning additional parameters. These innovations closely link CLUSTSEG to EM clustering and make it a transparent and powerful framework that yields superior results across the above segmentation tasks.
BibTeX
@inproceedings{icml2023_clustsegclusteri,
title = {CLUSTSEG: Clustering for Universal Segmentation},
author = {James Chenhao Liang and Tianfei Zhou and Dongfang Liu and Wenguan Wang},
booktitle = {ICML 2023},
year = {2023}
}