2024
NOLA: Compressing LoRA using Linear Combination of Random Basis
ICLR 2024poster
Fine-tuning Large Language Models (LLMs) and storing them for each downstream task or domain is impractical because of the massive model size (e.g., 350GB in GPT-3). Current literature, such as LoRA, showcases the potential of low-rank modifications to the original weights of an LLM, enabling effici…