Anastasia Razdaibiedina — accepted AI-conference papers

2023

Progressive Prompts: Continual Learning for Language Models

Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, Amjad Almahairi

ICLR 2023poster

We introduce Progressive Prompts – a simple and efficient approach for continual learning in language models. Our method allows forward transfer and resists catastrophic forgetting, without relying on data replay or a large number of task-specific parameters. Progressive Prompts learns a new soft pr…

Cited by 161 Source ↗PDF ↗code ↗Scholar ↗

2023

Representation Projection Invariance Mitigates Representation Collapse

Anastasia Razdaibiedina, Ashish Khetan, Zohar Karnin, Daniel Khashabi, Vivek Madan

EMNLP 2023long findings

Fine-tuning contextualized representations learned by pre-trained language models remains a prevalent practice in NLP. However, fine-tuning can lead to representation degradation (also known as representation collapse), which may result in instability, sub-optimal performance, and weak generalizatio…

Cited by 0 Source ↗Scholar ↗