Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency
Van-Anh Nguyen, Trung Le, Mehrtash Harandi, Ehsan Abbasnejad, Thanh-Toan Do, Dinh Phung
Abstract
We propose a framework grounded in gradient flow theory and informed by geometric structure that provides multiple diverse solutions for a given task, ensuring collaborative results that enhance performance and adaptability across different tasks. This framework enables flexibility, allowing for efficient task-specific fine-tuning while preserving the knowledge of the pre-trained foundation models. Extensive experiments across transfer learning, few-shot learning, and domain generalization show that our proposed approach consistently outperforms existing Bayesian methods, delivering strong performance with affordable computational overhead and offering a practical solution by updating only a small subset of parameters.
BibTeX
@inproceedings{
nguyen2025geometryaware,
title={Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency},
author={Van-Anh Nguyen and Trung Le and Mehrtash Harandi and Ehsan Abbasnejad and Thanh-Toan Do and Dinh Phung},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=SbJAGSdLhY}
}