Yuanfang Zhang

SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture

Despite the recent efforts from the NLP community, balancing the training budget, downstream performance, and general capabilities of large language models (LLM) remains a challenge in many applications. Training the entire model for downstream tasks is expensive, and could easily result in catastro…