2025
EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models
ICLR 2025poster
Distributed training methods are crucial for large language models (LLMs). However, existing distributed training methods often suffer from communication bottlenecks, stragglers, and limited elasticity, particularly in heterogeneous or large-scale environments. Local SGD methods have been proposed t…