Restabilizing Diffusion Models with Predictive Noise Fusion Strategy for Image Super-Resolution
Luoqian Jiang, Yong Guo, Bingna Xu, Haolin Pan, Jiezhang Cao, Wenbo Li, Jian Chen
Abstract
Diffusion models are prominent in image generation for producing detailed and realistic images from Gaussian noises. However, they often encounter instability issues in image restoration tasks, e.g., super-resolution. Existing methods typically rely on multiple runs to find an initial noise that produces a reasonably restored image. Unfortunately, these methods are computationally expensive and time-consuming without guaranteeing stable and consistent performance. To address these challenges, we propose a novel Predictive Noise Fusion Strategy (PNFS) that predicts pixel-wise errors in the restored image and combines different noises to generate a more effective noise. Extensive experiments show that PNFS significantly improves the stability and performance of diffusion models in super-resolution, both quantitatively and qualitatively. Furthermore, PNFS can be flexibly integrated into various diffusion models to enhance their stability.
BibTeX
@article{Jiang_Guo_Xu_Pan_Cao_Li_Chen_2025, title={Restabilizing Diffusion Models with Predictive Noise Fusion Strategy for Image Super-Resolution}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/32421}, DOI={10.1609/aaai.v39i4.32421}, abstractNote={Diffusion models are prominent in image generation for producing detailed and realistic images from Gaussian noises. However, they often encounter instability issues in image restoration tasks, e.g., super-resolution. Existing methods typically rely on multiple runs to find an initial noise that produces a reasonably restored image. Unfortunately, these methods are computationally expensive and time-consuming without guaranteeing stable and consistent performance. To address these challenges, we propose a novel Predictive Noise Fusion Strategy (PNFS) that predicts pixel-wise errors in the restored image and combines different noises to generate a more effective noise. Extensive experiments show that PNFS significantly improves the stability and performance of diffusion models in super-resolution, both quantitatively and qualitatively. Furthermore, PNFS can be flexibly integrated into various diffusion models to enhance their stability.}, number={4}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Jiang, Luoqian and Guo, Yong and Xu, Bingna and Pan, Haolin and Cao, Jiezhang and Li, Wenbo and Chen, Jian}, year={2025}, month={Apr.}, pages={4021-4029} }