Huang Xia

Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems

We propose Heterogeneous Swarms, an algorithm to design multi-LLM systems by jointly optimizing model roles and weights. We represent multi-LLM systems as directed acyclic graphs (DAGs) of LLMs with topological message passing for collaborative generation. Given a pool of LLM experts and a utility f…