Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

Vishvak Murahari; Dhruv Batra; Devi Parikh; Abhishek Das

ECCV 2020poster145 citations

Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

Vishvak Murahari, Dhruv Batra, Devi Parikh, Abhishek Das

Abstract

Prior work in visual dialog has focused on training deep neural models on VisDial in isolation. Instead, we present an approach to leverage pretraining on related vision-language datasets before transferring to visual dialog. We adapt the recently proposed ViLBERT model (Lu et al. 2019) for multi-turn visually-grounded conversations. Our model is pretrained on the Conceptual Captions and Visual Question Answering datasets, and finetuned on VisDial. Our best single model outperforms prior published work (including model ensembles) by more than 1% absolute on NDCG and MRR.

BibTeX

@inproceedings{eccv2020_largescalepretra,
  title = {Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline},
  author = {Vishvak Murahari and Dhruv Batra and Devi Parikh and Abhishek Das},
  booktitle = {ECCV 2020},
  year = {2020}
}