Sebastian Lee — accepted AI-conference papers

2025

A Theory of Initialisation's Impact on Specialisation

Devon Jarvis, Sebastian Lee, Clémentine Carla Juliette Dominé, Andrew M Saxe, Stefano Sarao Mannelli

ICLR 2025poster

Prior work has demonstrated a consistent tendency in neural networks engaged in continual learning tasks, wherein intermediate task similarity results in the highest levels of catastrophic interference. This phenomenon is attributed to the network's tendency to reuse learned features across tasks. H…

Cited by 0 Source ↗PDF ↗Scholar ↗

2022

Maslow’s Hammer in Catastrophic Forgetting: Node Re-Use vs. Node Activation

Sebastian Lee, Stefano Sarao Mannelli, Claudia Clopath, Sebastian Goldt, Andrew Saxe

ICML 2022spotlight

Continual learning—learning new tasks in sequence while maintaining performance on old tasks—remains particularly challenging for artificial neural networks. Surprisingly, the amount of forgetting does not increase with the dissimilarity between the learned tasks, but appears to be worst in an inter…

Cited by 17 Source ↗PDF ↗code ↗Scholar ↗

2021

Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

Sebastian Lee, Sebastian Goldt, Andrew Saxe

ICML 2021spotlight

Continual learning{—}the ability to learn many tasks in sequence{—}is critical for artificial learning systems. Yet standard training methods for deep networks often suffer from catastrophic forgetting, where learning new tasks erases knowledge of the earlier tasks. While catastrophic forgetting lab…

Cited by 94 Source ↗PDF ↗code ↗Scholar ↗