Peyman Milanfar — accepted AI-conference papers

2025

Kernel Density Steering: Inference-Time Scaling via Mode Seeking for Image Restoration

Yuyang Hu, Kangfu Mei, Mojtaba Sahraee-Ardakan, Ulugbek S. Kamilov, Peyman Milanfar, Mauricio Delbracio

NeurIPS 2025poster

Diffusion models show promise for image restoration, but existing methods often struggle with inconsistent fidelity and undesirable artifacts. To address this, we introduce Kernel Density Steering (KDS), a novel inference-time framework promoting robust, high-fidelity outputs through explicit local…

Cited by 0 Source ↗Scholar ↗

2025

Stochastic Deep Restoration Priors for Imaging Inverse Problems

Yuyang Hu, Albert Peng, Weijie Gan, Peyman Milanfar, Mauricio Delbracio, Ulugbek S. Kamilov

ICML 2025poster

Deep neural networks trained as image denoisers are widely used as priors for solving imaging inverse problems. We introduce Stochastic deep Restoration Priors (ShaRP), a novel framework that stochastically leverages an ensemble of deep restoration models beyond denoisers to regularize inverse probl…

Cited by 5 Source ↗PDF ↗Scholar ↗

2025

The Power of Context: How Multimodality Improves Image Super-Resolution

Kangfu Mei, Hossein Talebi, Mojtaba Ardakani, Vishal M. Patel, Peyman Milanfar, Mauricio Delbracio

CVPR 2025poster

Single-image super-resolution (SISR) remains challenging due to the inherent difficulty of recovering fine-grained details and preserving perceptual quality from low-resolution inputs. Existing methods often rely on limited image priors, leading to suboptimal results. We propose a novel approach tha…

Cited by 2 Source ↗PDF ↗Scholar ↗

2025

UniRes: Universal Image Restoration for Complex Degradations

Mo Zhou, Keren Ye, Mauricio Delbracio, Peyman Milanfar, Vishal M. Patel, Hossein Talebi

ICCV 2025poster

Real-world image restoration is hampered by diverse degradations stemming from varying capture conditions, capture devices and post-processing pipelines. Existing works make improvements through simulating those degradations and leveraging image generative priors, however generalization to in-the-wi…

Cited by 0 Source ↗PDF ↗Scholar ↗

2024

A Restoration Network as an Implicit Prior

Yuyang Hu, Mauricio Delbracio, Peyman Milanfar, Ulugbek Kamilov

ICLR 2024poster

Image denoisers have been shown to be powerful priors for solving inverse problems in imaging. In this work, we introduce a generalization of these methods that allows any image restoration network to be used as an implicit prior. The proposed method uses priors specified by deep neural networks pre…

Cited by 15 Source ↗PDF ↗Scholar ↗

2024

ArtVLM: Attribute Recognition Through Vision-Based Prefix Language Modeling

William Yicheng Zhu*, Keren Ye*, Junjie Ke, Jiahui Yu, Leonidas Guibas, Peyman Milanfar +1

ECCV 2024poster

"Recognizing and disentangling visual attributes from objects is a foundation to many computer vision applications. While large vision-language representations like CLIP had largely resolved the task of zero-shot object recognition, zero-shot visual attribute recognition remains a challenge because…

Cited by 2 Source ↗PDF ↗code ↗Scholar ↗

2024

CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation

Kangfu Mei, Mauricio Delbracio, Hossein Talebi, Zhengzhong Tu, Vishal M. Patel, Peyman Milanfar

CVPR 2024poster

Large generative diffusion models have revolutionized text-to-image generation and offer immense potential for conditional generation tasks such as image enhancement restoration editing and compositing. However their widespread adoption is hindered by the high computational cost which limits their r…

Cited by 13 Source ↗PDF ↗code ↗Scholar ↗

2024

Prompt-tuning Latent Diffusion Models for Inverse Problems

Hyungjin Chung, Jong Chul Ye, Peyman Milanfar, Mauricio Delbracio

ICML 2024poster

We propose a new method for solving imaging inverse problems using text-to-image latent diffusion models as general priors. Existing methods using latent diffusion models for inverse problems typically rely on simple null text prompts, which can lead to suboptimal performance. To improve upon this,…

Cited by 33 Source ↗PDF ↗Scholar ↗

2024

SPIRE: Semantic Prompt-Driven Image Restoration

Chenyang QI*, Zhengzhong Tu, Keren Ye, Mauricio Delbracio, Peyman Milanfar, Qifeng Chen +1

ECCV 2024poster

"Text-driven diffusion models have become increasingly popular for various image editing tasks, including inpainting, stylization, and object replacement. However, it still remains an open research problem to adopt this language-vision paradigm for more fine-level image processing tasks, such as den…

Cited by 4 Source ↗PDF ↗Scholar ↗

2023

MULLER: Multilayer Laplacian Resizer for Vision

Zhengzhong Tu, Peyman Milanfar, Hossein Talebi

ICCV 2023poster

Image resizing operation is a fundamental preprocessing module in modern computer vision. Throughout the deep learning revolution, researchers have overlooked the potential of alternative resizing methods beyond the commonly used resizers that are readily available, such as nearest-neighbors, biline…

Cited by 8 PDF ↗code ↗Scholar ↗

2023

Multiscale Structure Guided Diffusion for Image Deblurring

Mengwei Ren, Mauricio Delbracio, Hossein Talebi, Guido Gerig, Peyman Milanfar

ICCV 2023poster

Diffusion Probabilistic Models (DPMs) have recently been employed for image deblurring, formulated as an image-conditioned generation process that maps Gaussian noise to the high-quality image, conditioned on the blurry input. Image-conditioned DPMs (icDPMs) have shown more realistic results than re…

Cited by 77 PDF ↗Scholar ↗

2023

SVDiff: Compact Parameter Space for Diffusion Fine-Tuning

Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, Feng Yang

ICCV 2023poster

Recently, diffusion models have achieved remarkable success in text-to-image generation, enabling the creation of high-quality images from text prompts and various conditions. However, existing methods for customizing these models are limited by handling multiple personalized subjects and the risk o…

Cited by 271 PDF ↗Scholar ↗

2023

VILA: Learning Image Aesthetics From User Comments With Vision-Language Pretraining

Junjie Ke, Keren Ye, Jiahui Yu, Yonghui Wu, Peyman Milanfar, Feng Yang

CVPR 2023poster

Assessing the aesthetics of an image is challenging, as it is influenced by multiple factors including composition, color, style, and high-level semantics. Existing image aesthetic assessment (IAA) methods primarily rely on human-labeled rating scores, which oversimplify the visual aesthetic informa…

Cited by 82 Source ↗PDF ↗code ↗Scholar ↗

2022

Deblurring via Stochastic Refinement

Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, Peyman Milanfar

CVPR 2022oral

Image deblurring is an ill-posed problem with multiple plausible solutions for a given input image. However, most existing methods produce a deterministic estimate of the clean image and are trained to minimize pixel-level distortion. These metrics are known to be poorly correlated with human percep…

Cited by 334 PDF ↗Scholar ↗

2022

Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and Extracting Them From 2D Renderings

Innfarn Yoo, Huiwen Chang, Xiyang Luo, Ondrej Stava, Ce Liu, Peyman Milanfar +1

CVPR 2022poster

Digital watermarking is widely used for copyright protection. Traditional 3D watermarking approaches or commercial software are typically designed to embed messages into 3D meshes, and later retrieve the messages directly from distorted/undistorted watermarked 3D meshes. However, in many cases, user…

Cited by 41 PDF ↗Scholar ↗

2022

Interpretable Unsupervised Diversity Denoising and Artefact Removal

Mangal Prakash, Mauricio Delbracio, Peyman Milanfar, Florian Jug

ICLR 2022spotlight

Image denoising and artefact removal are complex inverse problems admitting multiple valid solutions. Unsupervised diversity restoration, that is, obtaining a diverse set of possible restorations given a corrupted image, is important for ambiguity removal in many applications such as microscopy wher…

Cited by 29 Source ↗PDF ↗Scholar ↗

2022

MAXIM: Multi-Axis MLP for Image Processing

Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik +1

CVPR 2022oral

Recent progress on Transformers and multi-layer perceptron (MLP) models provide new network architectural designs for computer vision tasks. Although these models proved to be effective in many vision tasks such as image recognition, there remain challenges in adapting them for low-level vision. The…

Cited by 624 PDF ↗code ↗Scholar ↗

2022

MaxViT: Multi-axis Vision Transformer

Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik +1

ECCV 2022poster

"Transformers have recently gained significant attention in the computer vision community. However, the lack of scalability of self-attention mechanisms with respect to image size has limited their wide adoption in state-of-the-art vision backbones. In this paper we introduce an efficient and scalab…

Cited by 916 Source ↗PDF ↗code ↗Scholar ↗

2021

COMISR: Compression-Informed Video Super-Resolution

Yinxiao Li, Pengchong Jin, Feng Yang, Ce Liu, Ming-Hsuan Yang, Peyman Milanfar

ICCV 2021poster

Most video super-resolution methods focus on restoring high-resolution video frames from low-resolution videos without taking into account compression. However, most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited. In this paper, we…

Cited by 48 PDF ↗code ↗Scholar ↗

2021

Learning To Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data

Abdullah Abuolaim, Mauricio Delbracio, Damien Kelly, Michael S. Brown, Peyman Milanfar

ICCV 2021poster

Recent work has shown impressive results on data-driven defocus deblurring using the two-image views available on modern dual-pixel (DP) sensors. One significant challenge in this line of research is access to DP data. Despite many cameras having DP sensors, only a limited number provide access to t…

Cited by 70 PDF ↗code ↗Scholar ↗

2021

Learning To Resize Images for Computer Vision Tasks

Hossein Talebi, Peyman Milanfar

ICCV 2021poster

For all the ways convolutional neural nets have revolutionized computer vision in recent years, one important aspect has received surprisingly little attention: the effect of image size on the accuracy of tasks being trained for. Typically, to be efficient, the input images are resized to a relative…

Cited by 170 PDF ↗Scholar ↗

2021

MUSIQ: Multi-Scale Image Quality Transformer

Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, Feng Yang

ICCV 2021poster

Image quality assessment (IQA) is an important research topic for understanding and improving visual experience. The current state-of-the-art IQA methods are based on convolutional neural networks (CNNs). The performance of CNN-based models is often compromised by the fixed shape constraint in batch…

Cited by 780 PDF ↗code ↗Scholar ↗

2021

Patch Craft: Video Denoising by Deep Modeling and Patch Matching

Gregory Vaksman, Michael Elad, Peyman Milanfar

ICCV 2021poster

The non-local self-similarity property of natural images has been exploited extensively for solving various image processing problems. When it comes to video sequences, harnessing this force is even more beneficial due to the temporal redundancy. In the context of image and video denoising, many cla…

Cited by 73 PDF ↗code ↗Scholar ↗

2021

Rich Features for Perceptual Quality Assessment of UGC Videos

Yilin Wang, Junjie Ke, Hossein Talebi, Joong Gon Yim, Neil Birkbeck, Balu Adsumilli +2

CVPR 2021poster

Video quality assessment for User Generated Content (UGC) is an important topic in both industry and academia. Most existing methods only focus on one aspect of the perceptual quality assessment, such as technical quality or compression artifacts. In this paper, we create a large scale dataset to co…

Cited by 105 PDF ↗Scholar ↗

2020

Distortion Agnostic Deep Watermarking

Xiyang Luo, Ruohan Zhan, Huiwen Chang, Feng Yang, Peyman Milanfar

CVPR 2020poster

Watermarking is the process of embedding information into an image that can survive under distortions, while requiring the encoded image to have little or no perceptual difference with the original image. Recently, deep learning-based methods achieved impressive results in both visual quality and me…

Cited by 233 PDF ↗Scholar ↗

2020

GIFnets: Differentiable GIF Encoding Framework

Innfarn Yoo, Xiyang Luo, Yilin Wang, Feng Yang, Peyman Milanfar

CVPR 2020poster

Graphics Interchange Format (GIF) is a widely used image file format. Due to the limited number of palette colors, GIF encoding often introduces color banding artifacts. Traditionally, dithering is applied to reduce color banding, but introducing dotted-pattern artifacts. To reduce artifacts and pro…

Cited by 7 PDF ↗Scholar ↗

2018

RED-UCATION: A Novel CNN Architecture Based on Denoising Nonlinearities

Yaniv Romano, Michael Elad, Peyman Milanfar

ICASSP 2018accepted

Image denoising is the most fundamental image enhancement task, and many algorithms have been proposed over the years for its solution. Interestingly, such an image denoising “engine” can be used to solve general inverse problems. Indeed, in our recent work we have presented the Regularization by De…

Cited by 0 Source ↗Scholar ↗