Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions

Tejas Jayashankar; Jongha Jon Ryu; Gregory W. Wornell

ICML 2025spotlight0 citations

Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions

Tejas Jayashankar, Jongha Jon Ryu, Gregory W. Wornell

Abstract

We propose *Score-of-Mixture Training* (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen–Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call *Score-of-Mixture Distillation* (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64×64 show that SMT/SMD are competitive with and can even outperform existing methods.

one-step generationskew Jensen-Shannon Divergencediffusion modelsscore estimation

BibTeX

@inproceedings{
jayashankar2025scoreofmixture,
title={Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions},
author={Tejas Jayashankar and Jongha Jon Ryu and Gregory W. Wornell},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=zk5k2NQcEA}
}