Learning Systems Expansion with Efficient Heterogeneity-aware Knowledge Transfer

Gaole Dai; Huatao Xu; Yifan Yang; Rui Tan; Mo Li

AAAI 2026technical0 citations

Learning Systems Expansion with Efficient Heterogeneity-aware Knowledge Transfer

Gaole Dai, Huatao Xu, Yifan Yang, Rui Tan, Mo Li

Abstract

Modern AI services must continually adapt to newly joined domains, yet delivering high-quality customized models is hampered by label sparsity, domain shifts, and tight budgets. We formulate this challenge as the learning system expansion problem and introduce HaT, an efficient heterogeneity-aware knowledge-transfer framework. HaT first selects a small set of high-quality source models with minimal overhead, and then fuses their imperfect predictions through a sample-wise attention mixer. Later, it adaptively distills the fused knowledge into target models via a knowledge dictionary. Extensive experiments on different tasks and modalities show that HaT outperforms state-of-the-art baselines by up to 16.5% accuracy, and saves 31.1% training time and up to 93.0% traffic.

BibTeX

@inproceedings{aaai2026_learningsystemse,
  title = {Learning Systems Expansion with Efficient Heterogeneity-aware Knowledge Transfer},
  author = {Gaole Dai and Huatao Xu and Yifan Yang and Rui Tan and Mo Li},
  booktitle = {AAAI 2026},
  year = {2026}
}