2026
Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos
ICML 2026poster
We develop a mean-field theory of dropout as a perturbation of critical signal propagation at the edge of chaos. Dropout shifts the perfect-alignment fixed point, making the depth scale for information propagation finite even at critical initialization. We derive critical and crossover scaling laws …