Jiawen Lyn

ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models

Parameter-efficient fine-tuning (PEFT) is widely studied for its effectiveness and efficiency in the era of large language models. Low-rank adaptation (LoRA) has demonstrated commendable performance as a popular and representative method. However, it is implemented with a fixed intrinsic rank that m…