Sahil Joshi — accepted AI-conference papers

2026

RACE Attention: A Strictly Linear-Time Attention for Long-Sequence Training

Sahil Joshi, Agniva Chowdhury, Amar Kanakamedala, Ekam Singh, Evan Tu, Anshumali Shrivastava

ICLR 2026poster

Softmax Attention has a quadratic time complexity in sequence length, which becomes prohibitive to run at long contexts, even with highly optimized GPU kernels. For example, FlashAttention-2/3 (exact, GPU-optimized implementations of Softmax Attention) cannot complete a single forward–backward pass…

Cited by 0 Source ↗code ↗Scholar ↗

2026

Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inference

Hoang Anh Duy Le, Sahil Joshi, Zeyu Yang, Zhaozhuo Xu, Anshumali Shrivastava

ICML 2026poster

Self-attention dominates the computational and memory cost of long-context LLM inference across both prefill and decode phases. To address this challenge, we introduce **Sketch\&Walk** Attention, a training-free sparse attention method that determines sparsity with lightweight sketches and determini…

Cited by 0 Source ↗Scholar ↗