Sketch-Based Low-Rank Model Merging with Shared Circulant Transforms
Merging multiple low-rank adapters (LoRA) provides a practical route to scaling multi-task learning and deployment more efficiently than full-model weight merging, while avoiding reliance on task-specific training data. However, most existing approaches either treat LoRA updates as dense weight delt…