COLING 2020main31 citations
Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements
Andrey Savchenko, Anton Alekseev, Sejeong Kwon, Elena Tutubalina, Evgeny Myasnikov, Sergey Nikolenko
Abstract
Understanding image advertisements is a challenging task, often requiring non-literal interpretation. We argue that standard image-based predictions are insufficient for symbolism prediction. Following the intuition that texts and images are complementary in advertising, we introduce a multimodal ensemble of a state of the art image-based classifier, a classifier based on an object detection architecture, and a fine-tuned language model applied to texts extracted from ads by OCR. The resulting system establishes a new state of the art in symbolism prediction.
BibTeX
@inproceedings{savchenko-etal-2020-ad,
title = "Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements",
author = "Savchenko, Andrey and
Alekseev, Anton and
Kwon, Sejeong and
Tutubalina, Elena and
Myasnikov, Evgeny and
Nikolenko, Sergey",
editor = "Scott, Donia and
Bel, Nuria and
Zong, Chengqing",
booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
month = dec,
year = "2020",
address = "Barcelona, Spain (Online)",
publisher = "International Committee on Computational Linguistics",
url = "https://aclanthology.org/2020.coling-main.171/",
doi = "10.18653/v1/2020.coling-main.171",
pages = "1886--1892"
}