Scaling Zero-Shot Reference-to-Video Generation
Zijian Zhou, Shikun Liu, Haozhe Liu, Haonan Qiu, Zhaochong An, Weiming Ren, Zhiheng Liu, Xiaoke Huang
Abstract
Reference-to-video (R2V) generation aims to synthesize videos that align with a text prompt while preserving the subject identity from reference images. However, current R2V methods are hindered by the reliance on explicit reference image-video-text triplets, whose construction is highly expensive and difficult to scale. We bypass this bottleneck by introducing Saber, a scalable zero-shot framework that requires no explicit R2V data. Trained exclusively on video-text pairs, Saber employs a masked training strategy and a tailored attention-based model design to learn identity-consistent and reference-aware representations. Mask augmentation techniques are further integrated to mitigate copy-paste artifacts common in reference-to-video generation. Moreover, Saber demonstrates remarkable generalization capabilities across a varying number of references and achieves superior performance on the OpenS2V-Eval benchmark compared to methods trained with R2V data.
BibTeX
@inproceedings{cvpr2026_scalingzeroshotr,
title = {Scaling Zero-Shot Reference-to-Video Generation},
author = {Zijian Zhou and Shikun Liu and Haozhe Liu and Haonan Qiu and Zhaochong An and Weiming Ren and Zhiheng Liu and Xiaoke Huang and Kam-Woh Ng and Tian Xie and Xiao Han and Yuren Cong and Hang Li and Chuyan Zhu and Aditya Patel and Tao Xiang and Sen He},
booktitle = {CVPR 2026},
year = {2026}
}