Learning Equi-angular Representations for Online Continual Learning
Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi
Abstract
Online continual learning suffers from an underfitted solution due to insufficient training for prompt model updates (e.g. single-epoch training). To address the challenge we propose an efficient online continual learning method using the neural collapse phenomenon. In particular we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100 TinyImageNet ImageNet-200 and ImageNet-1K we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaussian scheduled continuous (i.e. boundary-free) data setups.
BibTeX
@inproceedings{cvpr2024_learningequiangu,
title = {Learning Equi-angular Representations for Online Continual Learning},
author = {Minhyuk Seo and Hyunseo Koh and Wonje Jeung and Minjae Lee and San Kim and Hankook Lee and Sungjun Cho and Sungik Choi and Hyunwoo Kim and Jonghyun Choi},
booktitle = {CVPR 2024},
year = {2024}
}