Theme-Enhanced Hard Negative Sample Mining for Open-Domain Question Answering

Abstract

Dense passage retrieval has become the mainstream method in the first stage of open-domain question answering, which usually adopts a bi-encoder structure to learn the dense representation of questions and passages for semantic matching. One of the main challenges currently is to effectively utilize more informative hard negatives. Many efforts have been made to address this challenge by reducing the discrepancy between training and inference and training expensive cross-encoder for mining hard negatives. However, none of these approaches consider the theme information regarding the candidate passages. In this paper, we propose an approach for theme-enhanced hard negative sample mining, called THNSM. Firstly, we employ a topic sampling approach to mine hard negatives that are related to the topic. Secondly, we introduce a straightforward yet effective method for generating even more challenging hard negative samples by partially fusing their features. Empirical results on popular open-domain QA datasets demonstrate that THNSM significantly outperforms previous state-of-the-art models.