SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation
Zhuguanyu Wu, Ruihao Gong, Yang Yong, Yushi Huang, Xiangyu Fan, Lei Yang, Dahua Lin, Xianglong Liu
Abstract
Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models. However, DMD-style training faces a structural bottleneck: the student-side auxiliary score network (the fake score) must closely track a continuously evolving generator. Updating the fake score too frequently increases training cost and can over-emphasize inner-loop tracking, while infrequent updates lead to tracking lag that destabilizes training and degrades generation consistency. To address this issue, we propose \textbf{Score Gradient Matching Distillation (SGMD)}. SGMD adopts a fake-score perspective by directly optimizing the fake score toward the teacher, while using teacher stop-gradient Fisher as a stable distribution-matching objective. We provide a gradient analysis that motivates this objective choice under ideal tracking. Building on this, SGMD introduces a pair of dual potentials: negative-residual (NR) for outer-loop correction and residual-contraction (RC) for inner-loop tracking. Empirically, compared to DMD, SGMD achieves an approximately $\sim 3\times$ training speedup and substantially improves motion dynamics for 4-step distilled models while preserving temporal consistency.
BibTeX
@inproceedings{
wu2026sgmd,
title={{SGMD}: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation},
author={Zhuguanyu Wu and Ruihao Gong and Yang Yong and Yushi Huang and Xiangyu Fan and Lei Yang and Dahua Lin and Xianglong Liu},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=melQSx2bVT}
}