Ebru Aydin Gol

Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence

Language models lack the notion of interchangeable tokens: symbols that are semantically equivalent yet distinct, such as bound variables in formal logic. This limitation prevents generalization to larger vocabularies and hinders the model's ability to recognize alpha-equivalence, where renaming bou…