2026
Learning to Remember, Learn, and Forget in Attention-Based Models
ICML 2026poster
In-Context Learning (ICL) in transformers acts as an online associative memory and is believed to underpin their high performance on complex sequence processing tasks. However, in gated linear attention models, this memory has a fixed capacity and is prone to interference, especially for long sequen…