← Search

Alex Rogozhnikov

2 accepted papers

2021

CAPE: Encoding Relative Positions with Continuous Augmented Positional Embeddings

NeurIPS 2021poster

Without positional information, attention-based Transformer neural networks are permutation-invariant. Absolute or relative positional embeddings are the most popular ways to feed Transformer models with positional information. Absolute positional embeddings are simple to implement, but suffer from…

Cited by 61SourcePDFScholar