Yeonjae Kim

SpareTrain: Fault-Tolerant LLM Training via Low-Cost Dual Modular Redundancy

Dual Modular Redundancy (DMR) is a highly effective mechanism for detecting silent data corruption (SDC)—a critical reliability concern in large language model (LLM) training—by executing each operation twice. However, its high computation overhead has prevented practical deployment at scale. In thi…