2026
Deconstructing Positional Information: From Attention Logits to Training Biases
ICLR 2026poster
Positional encodings, a mechanism for incorporating sequential information into the Transformer model, are central to contemporary research on neural architectures. Previous work has largely focused on understanding their function through the principle of distance attenuation, where proximity dictat…