S M Rafiuddin

2 accepted papers

2026

Emergent Discrete Controller Modules for Symbolic Planning in Transformers

ICLR 2026poster

Transformers struggle with tasks that require symbolic planning loops, variable updates, and conditional branching, especially under length extrapolation. We introduce discrete controller modules that insert a small set of program primitives (ASSIGN, ADD, COMPARE, BRANCH) into Transformer blocks via…

Cited by 0 Source ↗Scholar ↗

2025

Learning What to Remember: Adaptive Probabilistic Memory Retention for Memory-Efficient Language Models

S M Rafiuddin, Muntaha Nujat Khan

EMNLP 2025

Transformer attention scales quadratically with sequence length O(n 2 ) , limiting long-context use. We propose Adaptive Retention , a probabilistic, layer-wise token selection mechanism that learns which representations to keep under a strict global budget M . Retention is modeled with Bernoulli ga

Cited by 0 Source ↗PDF ↗Scholar ↗