2026
Identifying and Mitigating Errors in Gradient Aggregation of Distributed Data Parallel Training
ICML 2026poster
Hardware-related silent data corruptions during gradient aggregation pose significant challenges to fault-tolerant distributed training, often leading to slow or failed convergence. To address this, we first mathematically formulate these errors as gradient inconsistency and theoretically analyze ho…