Learning the RoPEs: Better 2D and 3D Position Encodings with STRING
Connor Schenck, Isaac Reid, Mithun George Jacob, Alex Bewley, Joshua Ainslie, David Rendleman, Deepali Jain, Mohit Sharma
Abstract
We introduce $\textbf{STRING}$: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large language models, via a unifying theoretical framework. Importantly, STRING still provides $\textbf{exact}$ translation invariance, including token coordinates of arbitrary dimensionality, whilst maintaining a low computational footprint. These properties are especially important in robotics, where efficient 3D token representation is key. We integrate STRING into Vision Transformers with RGB(-D) inputs (color plus optional depth), showing substantial gains, e.g. in open-vocabulary object detection and for robotics controllers. We complement our experiments with a rigorous mathematical analysis, proving the universality of our methods. Videos of STRING-based robotics controllers can be found here: https://sites.google.com/view/string-robotics.
BibTeX
@inproceedings{
schenck2025learning,
title={Learning the Ro{PE}s: Better 2D and 3D Position Encodings with {STRING}},
author={Connor Schenck and Isaac Reid and Mithun George Jacob and Alex Bewley and Joshua Ainslie and David Rendleman and Deepali Jain and Mohit Sharma and Kumar Avinava Dubey and Ayzaan Wahid and Sumeet Singh and Ren{\'e} Wagner and Tianli Ding and Chuyuan Fu and Arunkumar Byravan and Jake Varley and Alexey A. Gritsenko and Matthias Minderer and Dmitry Kalashnikov and Jonathan Tompson and Vikas Sindhwani and Krzysztof Marcin Choromanski},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=XXFBqfwnUp}
}