Jacob Matthews

Semantics or spelling? Probing contextual word embeddings with orthographic noise

Pretrained language model (PLM) hidden states are frequently employed as contextual word embeddings (CWE): high-dimensional representations that encode semantic information given linguistic context. Across many areas of computational linguistics research, similarity between CWEs is interpreted as se…