DuetMerging: Synergizing Dynamic and Static Strategies for Mitigating Task Interference in Model Merging
Model merging offers a promising paradigm for consolidating multiple expert models into a single multitask architecture. However, its effectiveness is often hindered by task interference, where conflicting parameter updates from different tasks degrade performance. While dynamic, Mixture-of-Experts