A Training-Free Framework for High-Fidelity Appearance Transfer via Diffusion Transformers
Shengrong Gu, Ye Wang, Song Wu, Rui Ma, Qian Wang, Lanjun Wang, Zili Yi
Abstract
Diffusion Transformers (DiTs) excel at generation, but their global self-attention makes controllable, reference-image-based editing a distinct challenge. Unlike U-Nets, naively injecting local appearance into a DiT can disrupt its holistic scene structure. We address this by proposing the first training-free framework specifically designed to tame DiTs for high-fidelity appearance transfer. Our core is a synergistic system that disentangles structure and appearance. We leverage high-fidelity inversion to establish a rich content prior for the source image, capturing its lighting and micro-textures. A novel attention-sharing mechanism then dynamically fuses purified appearance features from a reference, guided by geometric priors. Our unified approach operates at 1024px and outperforms specialized methods on tasks ranging from semantic attribute transfer to fine-grained material application. Extensive experiments confirm our state-of-the-art performance in both structural preservation and appearance fidelity.
BibTeX
@inproceedings{icassp2026_atrainingfreefra,
title = {A Training-Free Framework for High-Fidelity Appearance Transfer via Diffusion Transformers},
author = {Shengrong Gu and Ye Wang and Song Wu and Rui Ma and Qian Wang and Lanjun Wang and Zili Yi},
booktitle = {ICASSP 2026},
year = {2026}
}