Julian G. Zilly

On Plasticity, Invariance, and Mutually Frozen Weights in Sequential Task Learning

Plastic neural networks have the ability to adapt to new tasks. However, in a continual learning setting, the configuration of parameters learned in previous tasks can severely reduce the adaptability to future tasks. In particular, we show that, when using weight decay, weights in successive layers…