Alejandro Fuster Baggetto

Is anisotropy really the cause of BERT embeddings not being semantic?

In this paper we conduct a set of experiments aimed to improve our understanding of the lack of semantic isometry in BERT, i.e. the lack of correspondence between the embedding and meaning spaces of its contextualized word representations. Our empirical results show that, contrary to popular belief,…