Andrej Jovanovic — accepted AI-conference papers

2026

DES-LOC: Desynced Low Communication Adaptive Optimizers for Foundation Models

Alex Iacob, Lorenzo Sani, Mher Safaryan, Paris Giampouras, Samuel Horváth, Meghdad Kurmanji +5

ICLR 2026poster

Scaling foundation model training with Distributed Data Parallel~(DDP) methods is bandwidth-limited. Existing infrequent communication methods like Local SGD were designed to synchronize model parameters only and cannot be trivially applied to adaptive optimizers due to additional optimizer states.…

Cited by 0 Source ↗Scholar ↗

2026

MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates

Alex Iacob, Andrej Jovanovic, Mher Safaryan, Meghdad Kurmanji, Lorenzo Sani, Samuel Horváth +3

ICLR 2026poster

Training large models with distributed data parallelism (DDP) requires frequent communication of gradients across workers, which can saturate bandwidth. Infrequent communication strategies (e.g., Local SGD) reduce this overhead but, when applied to adaptive optimizers, often suffer a performance gap…

Cited by 0 Source ↗Scholar ↗