Multi-Teacher Distillation for Incremental Object Detection

Abstract

Data replay and knowledge distillation are the most effective techniques for mitigating catastrophic forgetting in incremental object detection (IOD). Despite the recent achievements of IOD, there has been limited exploration regarding active learning of previously learned data, model plasticity, and industrial-grade incremental detectors. In this paper, we propose a multi-teacher distillation (MTD) method for the incremental learning of industrial detectors. Our proposed method leverages structural similarity loss to identify the most representative data, enhancing the efficiency of the incremental learning process. Additionally, we introduce an expert model that trained on the new data to alleviate the suppression from the teacher model on new categories. One of the key advantages of our method is its ability to significantly improve the performance on newly learned categories, especially when the amount of new data is limited. Through extensive experiments conducted on the PASCAL VOC and MS COCO benchmarks, we demonstrate that our proposed method achieves state-of-the-art performance, while ensuring both model stability and plasticity.