2026
Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway
ICML 2026poster
Recent theoretical analyses of multi-pathway Deep Linear Networks, typically grounded in Gradient Flow, predict a "winner-takes-all" specialization in which path symmetry breaks and each feature concentrates in a single pathway. In this work, we show that discrete Gradient Descent with a large step …