Tugrul Konuk

Tied-LoRA: Enhancing parameter efficiency of LoRA with Weight Tying

We introduce Tied-LoRA, a novel paradigm leveraging weight tying and selective training to enhance the parameter efficiency of Low-rank Adaptation (LoRA). Our exploration encompasses different plausible combinations of parameter training and freezing, coupled with weight tying, aimed at identifying…