2025
Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts
ICLR 2025poster
We present Self-MoE, an approach that transforms a monolithic LLM into a compositional, modular system of self-specialized experts, named MiXSE (MiXture of Self-specialized Experts). Our approach leverages self-specialization, which constructs expert modules using self-generated synthetic data, each…