2025
In-Context Denoising with One-Layer Transformers: Connections between Attention and Associative Memory Retrieval
ICML 2025oral
We introduce in-context denoising, a task that refines the connection between attention-based architectures and dense associative memory (DAM) networks, also known as modern Hopfield networks. Using a Bayesian framework, we show theoretically and empirically that certain restricted denoising problem…