Siyang Deng

Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs

Positional bias in large language models hinders their ability to effectively process long inputs. A prominent example is the “lost in the middle” phenomenon, where LLMs struggle to utilize relevant information situated in the middle of the input. While prior research primarily focuses on single pie…