Hunjae Lee

Dynamic Relational Priming Improves Transformer in Multivariate Time Series

Standard attention mechanisms in transformers employ static token representations that remain unchanged across all pair-wise computations in each layer. This limits their representational alignment with the potentially diverse dynamics of each token-pair interaction. While they excel in domains with…