Mingjie Liao

AC-ODM: Actor–Critic Online Data Mixing for Sample-Efficient LLM Pretraining

Optimizing pretraining data composition is pivotal for LLM generalization. While dynamic mixing outperforms static strategies by capturing evolving training dynamics, current methods fail to reconcile computational efficiency with sample efficiency and structural flexibility for diverse pipelines.We…