Visual Abstraction: A Plug-and-Play Approach for Text-Visual Retrieval
GuofengDing, Yiding Lu, Peng Hu, Mouxing Yang, Yijie Lin, Xi Peng
Abstract
Text-to-visual retrieval often struggles with semantic redundancy and granularity mismatches between textual queries and visual content. Unlike existing methods that address these challenges during training, we propose VISual Abstraction (VISA), a test-time approach that enhances retrieval by transforming visual content into textual descriptions using off-the-shelf large models. The generated text descriptions, with their dense semantics, naturally filter out low-level redundant visual information. To further address granularity issues, VISA incorporates a question-answering process, enhancing the text description with the specific granularity information requested by the user. Extensive experiments demonstrate that VISA brings substantial improvements in text-to-image and text-to-video retrieval for both short- and long-context queries, offering a plug-and-play enhancement to existing retrieval systems.
BibTeX
@inproceedings{
guofengding2025visual,
title={Visual Abstraction: A Plug-and-Play Approach for Text-Visual Retrieval},
author={GuofengDing and Yiding Lu and Peng Hu and Mouxing Yang and Yijie Lin and Xi Peng},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=DDvInyGVuz}
}