Brian Gallagher

RankMean: Module-Level Importance Score for Merging Fine-tuned LLM Models

Traditionally, developing new language models (LMs) capable of addressing multiple tasks involves fine-tuning pre-trained LMs using a wide collection of datasets, a process that often incurs significant computational expenses. Model merging emerges as a cost-effective alternative, allowing the integ…