Ziyun Qiao

GEM: Geometric Entropy Mixing for Optimal LLM Data Curation

LLM pre-training efficacy increasingly depends on data composition rather than sheer volume. Yet, optimal mixing is hindered by categorization flaws: human taxonomies suffer from ontological misalignment, and Euclidean clustering fails to address embedding anisotropy. We introduce **GEM** (**G**eome…