Justin Chih-Yao Chen

Symbolic Mixture-of-Experts: Adaptive Skill-based Routing for Heterogeneous Reasoning

Combining existing pre-trained LLMs is a promising avenue for tackling diverse reasoning tasks. However, selecting experts at the task level is often too coarse-grained, as heterogeneous tasks may require different expertise for each instance. To enable instance-level mixing of LLM experts, we propo…