AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation
Jingkun An, Yinghao Zhu, Zongjian Li, Enshen Zhou, Haoran Feng, Xijie Huang, Bohua Chen, Yemin Shi
Abstract
Text-to-Image (T2I) diffusion models have achieved remarkable success in image generation. Despite their progress, challenges remain in both prompt-following ability, image quality and lack of high-quality datasets, which are essential for refining these models. As acquiring labeled data is costly, we introduce AGFSync, a framework that enhances T2I diffusion models through Direct Preference Optimization (DPO) in a fully AI-driven approach. AGFSync utilizes Vision-Language Models (VLM) to assess image quality across style, coherence, and aesthetics, generating feedback data within an AI-driven loop. By applying AGFSync to leading T2I models such as SD v1.4, v1.5, and SDXL-base, our extensive experiments on the TIFA dataset demonstrate notable improvements in VQA scores, aesthetic evaluations, and performance on the HPS v2 benchmark, consistently outperforming the base models. AGFSync's method of refining T2I diffusion models paves the way for scalable alignment techniques.
BibTeX
@article{An_Zhu_Li_Zhou_Feng_Huang_Chen_Shi_Pan_2025, title={AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/32168}, DOI={10.1609/aaai.v39i2.32168}, abstractNote={Text-to-Image (T2I) diffusion models have achieved remarkable success in image generation. Despite their progress, challenges remain in both prompt-following ability, image quality and lack of high-quality datasets, which are essential for refining these models. As acquiring labeled data is costly, we introduce AGFSync, a framework that enhances T2I diffusion models through Direct Preference Optimization (DPO) in a fully AI-driven approach. AGFSync utilizes Vision-Language Models (VLM) to assess image quality across style, coherence, and aesthetics, generating feedback data within an AI-driven loop. By applying AGFSync to leading T2I models such as SD v1.4, v1.5, and SDXL-base, our extensive experiments on the TIFA dataset demonstrate notable improvements in VQA scores, aesthetic evaluations, and performance on the HPS v2 benchmark, consistently outperforming the base models. AGFSync’s method of refining T2I diffusion models paves the way for scalable alignment techniques.}, number={2}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={An, Jingkun and Zhu, Yinghao and Li, Zongjian and Zhou, Enshen and Feng, Haoran and Huang, Xijie and Chen, Bohua and Shi, Yemin and Pan, Chengwei}, year={2025}, month={Apr.}, pages={1746-1754} }