Ishita Khatri

Minibatch selection for Language Models via Partition Matroid Constrained Gradient Matching

Training Large Language Models (LLMs) on heterogeneous datasets requires optimizing domain representations to balance convergence speed and domain coverage. While recent methods reduce computational overhead by selecting high-quality data subsets, they typically perform selection independently per d…