2026
Ordered Local Momentum for Asynchronous Distributed Learning Under Arbitrary Delays
AAAI 2026technical
Momentum SGD (MSGD) serves as a foundational optimizer in training deep models due to momentum
4 accepted papers
Momentum SGD (MSGD) serves as a foundational optimizer in training deep models due to momentum
Byzantine-robust distributed learning (BRDL), which refers to distributed learning that can work with potential faulty or malicious workers (also known as Byzantine workers), has recently attracted much research attention. Robust aggregators are widely used in existing BRDL methods to obtain robustn…
Byzantine-robust distributed learning (BRDL), in which computing devices are likely to behave abnormally due to accidental failures or malicious attacks, has recently become a hot research topic. However, even in the independent and identically distributed (i.i.d.) case, existing BRDL methods will s…
Distributed learning is essential for training large-scale deep models. Asynchronous SGD (ASGD) and its variants are commonly used distributed learning methods, particularly in scenarios where the computing capabilities of workers in the cluster are heterogeneous. Momentum has been acknowledged for…