2026
Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
ICML 2026poster
Low-rank adaptation (LoRA) enables parameter-efficient specialization of foundation models, but the proliferation of task-specific adapters fragments capabilities across many adapters, complicating reuse and deployment. We study the problem of merging $T$ LoRAs into **a single rank-$r$ LoRA**, there…