Agglomerative Token Clustering
Joakim Bruslund Haurum*, Sergio Escalera, Graham W. Taylor*, Thomas B. Moeslund
Abstract
"We present Agglomerative Token Clustering (ATC), a novel token merging method that consistently outperforms previous token merging and pruning methods across image classification, image synthesis, and object detection & segmentation tasks. ATC merges clusters through bottom-up hierarchical clustering, without the introduction of extra learnable parameters. We find that ATC achieves state-of-the-art performance across all tasks, and can even perform on par with prior state-of-the-art when applied off-the-shelf, without fine-tuning. ATC is particularly effective when applied with low keep rates, where only a small fraction of tokens are kept and retaining task performance is especially difficult."
BibTeX
@inproceedings{eccv2024_agglomerativetok,
title = {Agglomerative Token Clustering},
author = {Joakim Bruslund Haurum* and Sergio Escalera and Graham W. Taylor* and Thomas B. Moeslund},
booktitle = {ECCV 2024},
year = {2024}
}