Self-Supervised Learning and Image-Prompt Fusion for AIGC Image Quality Assessment

Abstract

With the rapid advancement of artificial intelligence, the field of Artificial Intelligence Generated Content (AIGC) has seen significant growth. As AI-generated images (AIGIs) become increasingly prevalent, the AIGC image quality assessment(AIGCIQA) has gained critical importance. However, traditional image quality assessment methods struggle to account for the complex relationship between generated images and their corresponding text prompts, leading to suboptimal performance in AIGCIQA tasks. Additionally, the scarcity of subjective annotation data within AIGCIQA datasets limits the effectiveness of deep learning models. To address these challenges, we propose a novel self-supervised task that utilizes intermediate image sequences generated during Text-to-Image (T2I) model generation as the pre-training data. We use the number of model iterations as pseudo-labels based on the positive correlation between image quality and the number of model iterations. Due to the relative coarseness of this pseudo-label as a supervised signal, we also introduce a linear interpolation method for optimization. Additionally, we designed a framework that effectively fuses image and text features. The framework considers the dynamic fading properties of intermediate image sequences similar to video clips. It utilizes a specialized text prompt template and extracts image features through a cross-attention mechanism, which significantly improves model performance. Experimental results show that the proposed method achieves state-of-the-art performance on three mainstream AIGCIQA datasets.