2025
SMT: Fine-Tuning Large Language Models with Sparse Matrices
ICLR 2025poster
Various parameter-efficient fine-tuning (PEFT) methods, including LoRA and its variants, have gained popularity for reducing computational costs. However, there is often an accuracy gap between PEFT approaches and full fine-tuning (FT), and this discrepancy has not yet been systematically explored.…