Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning

Baolin Peng; Xiujun Li; Jianfeng Gao; Jingjing Liu; Yun-Nung Chen; Kam-Fai Wong

ICASSP 2018accepted0 citations

Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning

Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, Yun-Nung Chen, Kam-Fai Wong

Abstract

This paper presents a new method - adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems. Inspired by generative adversarial networks (GAN), we train a discriminator to differentiate responses/actions generated by dialogue agents from responses/actions by experts. Then, we incorporate the discriminator as another critic into the advantage actor-critic (A2C) framework, to encourage the dialogue agent to explore state-action within the regions where the agent takes actions similar to those of the experts. Experimental results in a movie-ticket booking domain show that the proposed Adversarial A2C can accelerate policy exploration efficiently.

BibTeX

@inproceedings{icassp2018_adversarialadvan,
  title = {Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning},
  author = {Baolin Peng and Xiujun Li and Jianfeng Gao and Jingjing Liu and Yun-Nung Chen and Kam-Fai Wong},
  booktitle = {ICASSP 2018},
  year = {2018}
}