2024
Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training
NeurIPS 2024poster
We explore the training dynamics of neural networks in a structured non-IID setting where documents are presented cyclically in a fixed, repeated sequence. Typically, networks suffer from catastrophic interference when training on a sequence of documents; however, we discover a curious and remarkabl…