Span Attention for Entity-Consistent Task-Oriented Dialogue Response Generation

Abstract

Task-oriented dialogue systems have recently gained increasing attention due to their capability of using natural language to fulfill specific user demands, such as restaurant reservation and hotel booking. Recent works directly model task-oriented dialogue response as a text generation task. However, these methods, utilizing generated response tokens as an attention query to obtain the vanilla attention distribution over an entire knowledge base, frequently lead to an entity inconsistency in final response generation. To tackle this problem, we propose a novel attention mechanism called span attention and a novel model named Span Attention GEnerator (SAGE). The span attention computes an attention score between a query vector and each knowledge record vector instead of computing a vanilla attention score among word vectors, which consisted of dialogue context and knowledge base. For effective training, we propose an attention constraint strategy that utilizes the entities appearing in response as pseudo-labels to supervise the training of the span attention. Experiments based on three publicly accessible datasets demonstrate that our model, utilizing the proposed mechanism, outperforms the state-of-the-art models with improvements of 8.94%, 2.29%, and 10.22% respectively in Entity F1.