ACL 2024short4 citations
Isotropy, Clusters, and Classifiers
Timothee Mickus, Stig-Arne Grönroos, Joseph Attieh
Abstract
Whether embedding spaces use all their dimensions equally, i.e., whether they are isotropic, has been a recent subject of discussion. Evidence has been accrued both for and against enforcing isotropy in embedding spaces. In the present paper, we stress that isotropy imposes requirements on the embedding space that are not compatible with the presence of clusters—which also negatively impacts linear classification objectives. We demonstrate this fact both empirically and mathematically and use it to shed light on previous results from the literature.
BibTeX
@inproceedings{mickus-etal-2024-isotropy,
title = "Isotropy, Clusters, and Classifiers",
author = {Mickus, Timothee and
Gr{\"o}nroos, Stig-Arne and
Attieh, Joseph},
editor = "Ku, Lun-Wei and
Martins, Andre and
Srikumar, Vivek",
booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
month = aug,
year = "2024",
address = "Bangkok, Thailand",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.acl-short.7/",
doi = "10.18653/v1/2024.acl-short.7",
pages = "75--84"
}