2026
Balancing the Experts: Unlocking LoRA-MoE for GRPO via Mechanism-Aware Rewards
ICLR 2026poster
Parameter-efficient Mixture-of-Experts (MoE) architectures, such as LoRA-MoE, enable strong and generalizable fine-tuning. However, a critical problem arises when fine-tuning these architectures with advanced reinforcement learning algorithms such as Group Relative Policy Optimization (GRPO). Tradit…