Yingming Zheng

When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models

Large language models (LLMs) have achieved impressive performance across natural language processing (NLP) tasks. As real-world applications increasingly demand longer context windows, continued pretraining and supervised fine-tuning (SFT) on long-context data has become a common approach. While the