2026
SPARKLING: Balancing Signal Preservation and Symmetry Breaking for Width-Progressive Learning
ICML 2026poster
Progressive Learning (PL) reduces pre-training computational overhead by gradually increasing model scale. While prior work has extensively explored depth expansion, width expansion remains significantly understudied, with the few existing methods limited to the early stages of training. However, ex…