Unsupervised Image-to-Image Style Transfer via Dual-Condition Diffusion Models*

Abstract

Style transfer is an artistic research topic within a series of generative tasks, and it is quite challenging to stably and reliably guide the generation of images with the expected content and style. Early methods followed a predefined style, defined by a reference image, and applied it to another image while preserving the latter’s content structure, but the generated effects lacked high fidelity. As diffusion models have evolved, the content guidance for style transfer has transitioned from images to text, potentially leading to instance alterations. Generating with multiple conditions constrained in the latent space is challenging. Within the EDM framework, we have utilized cross-attention to design style and content embedding modules, overcoming instance alterations without the need for paired datasets and achieving high-resolution, high-fidelity results. Our experimental findings demonstrate that our architecture has advanced to the forefront of image-to-image style transfer capabilities.