A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment
Tianhe Wu, Kede Ma*, Jie Liang, Yujiu Yang*, Lei Zhang
Abstract
"While Multimodal Large Language Models (MLLMs) have experienced significant advancement in visual understanding and reasoning, their potential to serve as powerful, flexible, interpretable, and text-driven models for Image Quality Assessment (IQA) remains largely unexplored. In this paper, we conduct a comprehensive and systematic study of prompting MLLMs for IQA. We first investigate nine prompting systems for MLLMs as the combinations of three standardized testing procedures in psychophysics (, the single-stimulus, double-stimulus, and multiple-stimulus methods) and three popular prompting strategies in natural language processing (, the standard, in-context, and chain-of-thought prompting). We then present a difficult sample selection procedure, taking into account sample diversity and uncertainty, to further challenge MLLMs equipped with the respective optimal prompting systems. We assess three open-source and one closed-source MLLMs on several visual attributes of image quality (, structural and textural distortions, geometric transformations, and color differences) in both full-reference and no-reference scenarios. Experimental results show that only the closed-source GPT-4V provides a reasonable account for human perception of image quality, but is weak at discriminating fine-grained quality variations (, color differences) and at comparing visual quality of multiple images, tasks humans can perform effortlessly."
BibTeX
@inproceedings{eccv2024_acomprehensivest,
title = {A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment},
author = {Tianhe Wu and Kede Ma* and Jie Liang and Yujiu Yang* and Lei Zhang},
booktitle = {ECCV 2024},
year = {2024}
}