Abdessalam Ed-dib

GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning

Fine-tuning large language models (LLMs) is computationally expensive because it requires updating all model parameters. Low-Rank Adaptation (LoRA) reduces this cost by modifying a subset of weights, but selecting the appropriate rank introduces a trade-off: lower ranks improve efficiency at the exp