Policy Adaptation for Deep Reinforcement Learning-Based Dialogue Management
Lu Chen, Cheng Chang, Zhi Chen, Bowen Tan, Milica Gasic, Kai Yu
Abstract
Policy optimization is the core part of statistical dialogue management. Deep reinforcement learning has been successfully used for dialogue policy optimization for a static pre-defined domain. However, when the domain changes dynamically, e.g. a new previously unseen concept (or slot) which can be then used as a database search constraint is added, or the policy for one domain is transferred to another domain, the dialogue state space and action sets both will change. Therefore, the model structures for different domains have to be different. This makes dialogue policy adaptation/transfer challenging. Here a multi -agent dialogue policy (MADP) is proposed to tackle these problems. MADP consists of some slot-dependent agents (S-Agents) and a slot-independent agent (G-Agent). S-Agents have shared parameters in addition to private parameters for each one. During policy transfer, the shared parameters in S-Agents and the parameters in G-Agent can be directly transferred to the agents in extended/new domain. Simulation experiments showed that MADP can significantly speed up the policy learning and facilitate policy adaptation.
BibTeX
@inproceedings{icassp2018_policyadaptation,
title = {Policy Adaptation for Deep Reinforcement Learning-Based Dialogue Management},
author = {Lu Chen and Cheng Chang and Zhi Chen and Bowen Tan and Milica Gasic and Kai Yu},
booktitle = {ICASSP 2018},
year = {2018}
}