2026
SpareTrain: Fault-Tolerant LLM Training via Low-Cost Dual Modular Redundancy
ICLR 2026poster
Dual Modular Redundancy (DMR) is a highly effective mechanism for detecting silent data corruption (SDC)—a critical reliability concern in large language model (LLM) training—by executing each operation twice. However, its high computation overhead has prevented practical deployment at scale. In thi…