Enhancing Data-Free Class-Incremental Learning via Image-Centric Dual Distillation

Abstract

In Data-free Class-Incremental Learning (DFCIL), catastrophic forgetting is a significant challenge due to the lack of access to previous task image data. Recent approaches using model inversion have made progress in addressing this issue, yet the suboptimal application of knowledge distillation hampers new task learning, limiting overall model performance. To overcome this, we propose a novel method incorporating image-centric dual distillation, designed to retain more old knowledge while facilitating new knowledge acquisition, thus enhancing DFCIL performance. Specifically, we first introduce a weak-constraint relation distillation strategy to preserve old knowledge while promoting the assimilation of new knowledge by learning the relationships among intra-class samples. Then, to further enhance the preservation of old knowledge and refine the integration of new knowledge, we introduce a low-level feature distillation strategy to retain foundational general knowledge by leveraging semantic information from shallow network layers. Extensive experiments show the effectiveness of our method.