Sjouke Mauw

How does the optimizer implicitly bias the model merging loss landscape?

Model merging methods combine models with different capabilities into a single one while maintaining the same inference cost. Two popular approaches are lin- ear interpolation, which linearly interpolates between model weights, and task arithmetic, which combines task vectors obtained by the differe…