CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics
Yiren Song, Xuning Shao, Kang Chen, Weidong Zhang, Zhongliang Jing, Minzhe Li
Abstract
Considerable progress has recently been made in leveraging CLIP (Contrastive Language-Image Pre-Training) models for text-guided image manipulation. However, all existing works rely on additional generative models to ensure the quality of results, because CLIP alone cannot provide enough guidance information for fine-scale pixel-level changes. In this paper, we introduce CLIPVG, a text-guided image manipulation framework using differentiable vector graphics, which is also the first CLIP-based general image manipulation framework that does not require any additional generative models. We demonstrate that CLIPVG can not only achieve state-of-art performance in both semantic correctness and synthesis quality, but also is flexible enough to support various applications far beyond the capability of all existing methods.
BibTeX
@article{Song_Shao_Chen_Zhang_Jing_Li_2023, title={CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/25326}, DOI={10.1609/aaai.v37i2.25326}, abstractNote={Considerable progress has recently been made in leveraging CLIP (Contrastive Language-Image Pre-Training) models for text-guided image manipulation. However, all existing works rely on additional generative models to ensure the quality of results, because CLIP alone cannot provide enough guidance information for fine-scale pixel-level changes. In this paper, we introduce CLIPVG, a text-guided image manipulation framework using differentiable vector graphics, which is also the first CLIP-based general image manipulation framework that does not require any additional generative models. We demonstrate that CLIPVG can not only achieve state-of-art performance in both semantic correctness and synthesis quality, but also is flexible enough to support various applications far beyond the capability of all existing methods.}, number={2}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Song, Yiren and Shao, Xuning and Chen, Kang and Zhang, Weidong and Jing, Zhongliang and Li, Minzhe}, year={2023}, month={Jun.}, pages={2312-2320} }