Weifeng.liu

Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps

Long-context language models (LCLMs), characterized by their extensive context window, are becoming popular. However, despite the fact that they are nearly perfect at standard long-context retrieval tasks, our evaluations demonstrate they fail in some basic cases. Later, we find they can be well add