3 accepted papers
Model merging methods combine models with different capabilities into a single one while maintaining the same inference cost. Two popular approaches are lin- ear interpolation, which linearly interpolates between model weights, and task arithmetic, which combines task vectors obtained by the differe…
Understanding the geometry of neural network loss landscapes is a central question in deep learning, with implications for generalization and optimization. A striking phenomenon is $\textit{linear mode connectivity}$ (LMC), where independently trained models can be connected by low- or zero-barrier…
Structural pruning of neural networks conventionally relies on identifying and discarding less important neurons, a practice often resulting in significant accuracy loss that necessitates subsequent fine-tuning efforts. This paper introduces a novel approach named Intra-Fusion, challenging this prev…