Rupert Mitchell — accepted AI-conference papers

2026

Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining

ICML 2026poster

We present Multipole Semantic Attention (MuSe), an efficient approximation of softmax attention for long-context transformers. MuSe clusters queries and keys separately in their learned representation spaces, computing query-specific cluster summaries that capture how each query cluster attends to e…

Cited by 0 Source ↗Scholar ↗

2025

Where is the Truth? The Risk of Getting Confounded in a Continual World

Florian Peter Busch, Roshni Ramanna Kamath, Rupert Mitchell, Wolfgang Stammer, Kristian Kersting, Martin Mundt

ICML 2025spotlight

A dataset is confounded if it is most easily solved via a spurious correlation which fails to generalize to new data. In this work, we show that, in a continual learning setting where confounders may vary in time across tasks, the challenge of mitigating the effect of confounders far exceeds the sta…

Cited by 6 Source ↗PDF ↗code ↗Scholar ↗