Linh Duy Tran — accepted AI-conference papers

2026

Expert Merging in Sparse Mixture of Experts with Nash Bargaining

Dung Viet Nguyen, Anh Nguyen Thi, Minh Hoang Nguyen, Luc Nguyen, Shiqi Jiang, Ethan Fetaya +3

ICLR 2026poster

Existing expert merging strategies for Sparse Mixture of Experts (SMoE) typically rely on input-dependent or input-independent averaging of expert parameters, but often lack a principled weighting mechanism. In this work, we reinterpret expert merging through the lens of game theory, revealing coope…

Cited by 0 Source ↗code ↗Scholar ↗

2025

CAMEx: Curvature-aware Merging of Experts

Viet Dung Nguyen, Minh Nguyen Hoang, Luc Nguyen, Rachel Teo, Tan Minh Nguyen, Linh Duy Tran

ICLR 2025poster

Existing methods for merging experts during model training and fine-tuning predominantly rely on Euclidean geometry, which assumes a flat parameter space. This assumption can limit the model's generalization ability, especially during the pre-training phase, where the parameter manifold might exhibi…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗