2025
Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
ICML 2025spotlight
Fine-tuning a pre-trained model on a downstream task often degrades its original capabilities, a phenomenon known as "catastrophic forgetting". This is especially an issue when one does not have access to the data and recipe used to develop the pre-trained model. Under this constraint, most existing…