DialogMI: A Dialogue Model Based on Enhancing Dialogue Mutual Information

Abstract

Most of the open-domain dialogue models tend to perform insufficiently in generating informative response. The possible reason is that they lack the capability of enhancing the mutual information between generated responses and dialogue history. To address this issue, we present a novel task of the mutual information enhancement and build a dialogue generation framework DialogMI. Besides, we propose two methods to represent the loss function of the novel task and a method to enhance the effect of the mutual information loss. To the best of our knowledge, we are the first to propose a loss that measures mutual information and use it to assist training model. We conduct experiments on Chinese chit-chat dataset and LCCC dataset for response generation. Results on these datasets indicate that Di-alogMI can significantly outperform baselines in generating informative dialogue, leading to better response quality and diversity.