MoE^2: A Mixture-of-Mixtures of Experts for Ensemble-Free Domain Generalization
Ahmed Radwan, Mahmoud Soliman, Omar Abdelaziz, Ahmad Abdel-Qader, Mohamed S. Shehata
Abstract
Domain Generalization (DG) requires models to generalize across unseen data distributions. Kernel-based theory reveals a No-Free-Lunch problem: any model with a fixed representation is fundamentally sub-optimal for all possible shifts. While large ensembles mitigate this, they are computationally expensive and remain static once trained, inheriting the same theoretical limitation. We introduce MoE² (Mixture-of-Mixtures of Experts), a framework that uses a single frozen backbone to dynamically synthesize a bespoke adapter for each input, allowing it to continuously adapt its effective kernel. We provide a theoretical grounding for this process, proving our routing mechanism is a principled non-parametric estimator for the optimal Bayes mixture of experts. We derive a generalization bound that cleanly separates the router
BibTeX
@inproceedings{aaai2026_moe2amixtureofmi,
title = {MoE^2: A Mixture-of-Mixtures of Experts for Ensemble-Free Domain Generalization},
author = {Ahmed Radwan and Mahmoud Soliman and Omar Abdelaziz and Ahmad Abdel-Qader and Mohamed S. Shehata},
booktitle = {AAAI 2026},
year = {2026}
}