2026
FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability–Plasticity Tradeoff
ICLR 2026oral
Deep neural networks trained on nonstationary data must balance stability (i.e., retaining prior knowledge) and plasticity (i.e., adapting to new tasks). Standard reinitialization methods, which reinitialize weights toward their original values, are widely used but difficult to tune: conservative re…