The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasure
Embedding-based similarity metrics between text sequences can be influenced not just by the content dimensions we most care about, but can also be biased by spurious attributes like the text’s source or language. These document confounders cause problems for many applications, but especially those t