2026
Not All Documents Are What You Need for Extracting Instruction Tuning Data
ICLR 2026poster
Instruction tuning improves the LLMs performance but depends on high-quality training data. Recently, LLMs have been used to synthesize data, enhancing training with seeds like question-answer (QA) pairs. However, this synthesis often results in instruction examples similar to the seeds, lacking div…