Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency
Retrieval-augmented language models have demonstrated performance comparable to much larger models while requiring fewer computational resources. The effectiveness of these models crucially depends on the overlap between query and retrieved context, but the optimal degree of this overlap remains une