REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning
Sungho Jeon, Xinyue Ma, Kwang In Kim, Myeongjae Jeon
Abstract
Recent rehearsal-free continual learning (CL) methods guided by prompts achieve strong performance on vision tasks with non-stationary data but remain resource-intensive, hindering real-world deployment. We introduce resource-efficient prompting (REP), which improves the computational and memory efficiency of prompt-based rehearsal-free methods while minimizing accuracy trade-offs. Our approach employs swift prompt selection to refine input data using a carefully provisioned model and introduces adaptive token merging AToM and adaptive layer dropping ALD for efficient prompt updates. AToM and ALD selectively skip data and model layers while preserving task-specific features during the learning of new tasks. Extensive experiments on multiple image classification datasets demonstrate REP’s superior resource efficiency over state-of-the-art rehearsal-free CL methods.
BibTeX
@inproceedings{
jeon2025rep,
title={{REP}: Resource-Efficient Prompting for Rehearsal-Free Continual Learning},
author={Sungho Jeon and Xinyue Ma and Kwang In Kim and Myeongjae Jeon},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=QjmRIgTcU8}
}