Andras Szecsenyi

Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modeling and State Tracking

State-space language models such as Mamba and gated linear attention (GLA) offer efficient alternatives to transformers due to their linear complexity and parallel training, but often lack the expressivity and robust state-tracking needed for complex reasoning. We address these limitations by refram…