CVPR 2016spotlight610 citations
Where to Look: Focus Regions for Visual Question Answering
Kevin J. Shih, Saurabh Singh, Derek Hoiem
Abstract
We present a method that learns to answer visual questions by selecting image regions relevant to the text-based query. Our method maps textual queries and visual features from various regions into a shared space where they are compared for relevance with an inner product. Our method exhibits significant improvements in answering questions such as "what color," where it is necessary to evaluate a specific location, and "what room," where it selectively identifies informative image regions. Our model is tested on the recently released VQA dataset, which features free-form human-annotated questions and answers.
BibTeX
@inproceedings{cvpr2016_wheretolookfocus,
title = {Where to Look: Focus Regions for Visual Question Answering},
author = {Kevin J. Shih and Saurabh Singh and Derek Hoiem},
booktitle = {CVPR 2016},
year = {2016}
}