Low-Rank Transformer Adaptation for Arbitrary Style Transfer

Abstract

Arbitrary style transfer aims to apply artistic characteristics from a style reference to an image while preserving the image’s original content. Although many methods have achieved remarkable results in style transfer, they typically rely on largescale datasets for training, which increases both the cost and complexity of data collection. To address this issue, we propose a Low-rank Transformer Adaptation method for style transfer which leverages the efficiency of low-rank adaptation to reduce the model’s complexity without compromising performance. It not only accelerates the training process but also delivers high-quality image generation with a small-scale dataset. Additionally, we introduce an edge detection loss to enhance the preservation of content outlines further, ensuring that the fine details of the image are maintained during the style transfer process Experimental results demonstrate that our method achieves competitive performance even with significantly less data, and it exhibits superiority in both visual quality and evaluation metrics.