2025
Rotary Masked Autoencoders are Versatile Learners
Uros Zivanovic, Serafina Di Gioia, Andre Scaffidi, Martín Emilio de los Rios, Gabriella Contardo, Roberto Trotta
NeurIPS 2025poster
Applying Transformers to irregular time-series typically requires specializations to their baseline architecture, which can result in additional computational overhead and increased method complexity. We present the Rotary Masked Autoencoder (RoMAE), which utilizes the popular Rotary Positional Embe…