2026
Learning in the Fisher Subspace: A Guided Initialization for LoRA Fine-Tuning
ICML 2026poster
LoRA adapts large language models (LLMs) by restricting updates to low-rank subspaces of pre-trained weights. While this substantially reduces training cost, the effectiveness of adaptation critically depends on which subspace is chosen at initialization: a poor initialization that allocates capacit…