kNN-CL: Enhancing Continual Learning with Nearest Neighbor Retrieval

Abstract

Continual learning aims to learn new tasks sequentially without forgetting previously acquired knowledge. However, catastrophic forgetting remains a significant challenge. In this paper, we introduce kNN-CL, a simple yet effective approach that harnesses k-nearest neighbors (kNN) to mitigate forgetting in continual learning. Specifically, kNN-CL identifies the k most similar instances (key-value pairs) from the previous tasks to refine model predictions, enabling the model to adapt to the relevant task for a given test instance. Notably, kNN-CL can be seamlessly integrated into existing continual learning frameworks in a plug-and-play manner, without any additional training. Modern deep neural networks have achieved remarkable progress on a wide range of tasks. However, they encounter difficulties when processing sequential data streams. As these networks recalibrate their parameters to assimilate new data, they inadvertently compromise on previously acquired knowledge, leading to an issue known as catastrophic forgetting. Experimental results show that kNN-CL substantially improves accuracy in both settings, demonstrating the effectiveness of kNN-CL in mitigating catastrophic forgetting.