2026
DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Training
ICLR 2026poster
Determinism is indispensable for reproducibility in large language model (LLM) training, yet it often exacts a steep performance cost. In widely used attention implementations such as FlashAttention-3, the deterministic backward pass can incur up to a 37.9% throughput reduction relative to its non‑d…