NeurIPS 2022accept215 citations
CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders
Kevin Frans, Lisa Soros, Olaf Witkowski
Abstract
CLIPDraw is an algorithm that synthesizes novel drawings from natural language input. It does not require any additional training; rather, a pre-trained CLIP language-image encoder is used as a metric for maximizing similarity between the given description and a generated drawing. Crucially, CLIPDraw operates over vector strokes rather than pixel images, which biases drawings towards simpler human-recognizable shapes. Results compare CLIPDraw with other synthesis-through-optimization methods, as well as highlight various interesting behaviors of CLIPDraw.
image synthesisclipcomputer visionlanguage to textcreativityart
BibTeX
@inproceedings{
frans2022clipdraw,
title={{CLIPD}raw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders},
author={Kevin Frans and Lisa Soros and Olaf Witkowski},
booktitle={Advances in Neural Information Processing Systems},
editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
year={2022},
url={https://openreview.net/forum?id=c39zYHHgQmy}
}