A New Model for Prototype-based Continual Learning in Hyperspherical Space

Abstract

The continuous emergence of new objects in the visual world poses a serious challenge to deep object recognition methods, which sparks the increasing study on continual or incremental learning. However, learning new tasks faces the tough catastrophic forgetting problem, i.e., dramatic performance degradation on old tasks. A good continual learning model should be robustly adapted to the upcoming tasks while effectively handling catastrophic forgetting. In this paper, we focus on the class-incremental learning (CIL) task, and propose a novel prototype-based continual learning model C-HPN that projects the visual features into a hypersphere geometric space, where continual learning is conducted. C-HPN features two-fold contributions. On the one hand, instead of using the popular cross-entropy loss, we develop an instance-prototype compact loss to obtain well-clustered hyperspherical embeddings and a prototype-prototype separability loss to boost the model’s generalization by introducing large angle distance inductive bias between prototypes in the hyperspherical space. On the other hand, prototype construction and adaptation strategies are designed for effectively adapting new classes, and an instance-prototype relationship preservation distillation mechanism is introduced to overcome catastrophic forgetting. Extensive experiments on several image datasets validate the effectiveness of the proposed method.