Precognition in Contextual Spoken Language Understanding via Knowledge Distillation
Nan Su, Bingzhu Du, Yuchi Zhang, Chao Liu, Yongliang Wang, Hong Chen, Xin Lu
Abstract
Task-oriented dialogue systems have become overwhelmingly popular in recent researches. Spoken Language Understanding (SLU) is widely used to extract the semantics frame of user queries and comprehend users’ intent/emotion/dialogue state in task-oriented dialogue systems. Most previous works on such discriminative tasks only model current query or historical conversations. Even if the entire conversation flow is modeled in some work, it is not suitable for real-world task-oriented dialogue systems, where future contexts are not visible until a response is given based on the current dialogue state. In this paper, we propose to jointly model historical and future information using knowledge distillation methods to address the discrepancy between offline and online information in dialogue understanding. More specifically, by modeling the current utterance and past contexts as student, and the entire dialogue flow as teacher, we optimize the "distance" between these two networks to regularize our student model in both online and offline ways. Our experiments on two dialogue datasets validate the effectiveness of our proposed method, achieving superior results compared with all baseline models.
BibTeX
@inproceedings{icassp2023_precognitioninco,
title = {Precognition in Contextual Spoken Language Understanding via Knowledge Distillation},
author = {Nan Su and Bingzhu Du and Yuchi Zhang and Chao Liu and Yongliang Wang and Hong Chen and Xin Lu},
booktitle = {ICASSP 2023},
year = {2023}
}