L-CoIns: Language-Based Colorization With Instance Awareness
Zheng Chang, Shuchen Weng, Peixuan Zhang, Yu Li, Si Li, Boxin Shi
Abstract
Language-based colorization produces plausible colors consistent with the language description provided by the user. Recent studies introduce additional annotation to prevent color-object coupling and mismatch issues, but they still have difficulty in distinguishing instances corresponding to the same object words. In this paper, we propose a transformer-based framework to automatically aggregate similar image patches and achieve instance awareness without any additional knowledge. By applying our presented luminance augmentation and counter-color loss to break down the statistical correlation between luminance and color words, our model is driven to synthesize colors with better descriptive consistency. We further collect a dataset to provide distinctive visual characteristics and detailed language descriptions for multiple instances in the same image. Extensive experiments demonstrate our advantages of synthesizing visually pleasing and description-consistent results of instance-aware colorization.
BibTeX
@inproceedings{cvpr2023_lcoinslanguageba,
title = {L-CoIns: Language-Based Colorization With Instance Awareness},
author = {Zheng Chang and Shuchen Weng and Peixuan Zhang and Yu Li and Si Li and Boxin Shi},
booktitle = {CVPR 2023},
year = {2023}
}