Alessandro Breccia

The Importance of Being Lazy: Scaling Limits of Continual Learning

Despite recent efforts, neural networks still struggle to learn in non-stationary environments, and our understanding of catastrophic forgetting (CF) is far from complete. In this work, we perform a systematic study on the impact of model scale and the degree of feature learning in continual learnin…