Generative Diffusion Prior for Unified Image and Video Restoration & Enhancement

Abstract Existing image restoration methods primarily rely on the posterior distribution of natural images but are often limited by their dependence on known degradations and supervised training. To this end, we propose Generative Diffusion Prior (GDP), an unsupervised sampling-based framework that effectively models posterior distributions for image and video restoration. GDP utilizes a single pre-trained denoising diffusion probabilistic model (DDPM) to solve a wide range of linear, non-linear, and blind inverse problems without explicit degradation assumptions. Specifically, GDP systematically explores a conditional guidance protocol, which proves more practical and effective than conventional methods of adding guidance. Furthermore, GDP incorporates a degradation model optimization mechanism during the denoising process, enabling blind image restoration. Besides, we introduce a patch-based strategy, allowing GDP to handle images of arbitrary resolution. We extensively evaluate GDP on multiple image and video restoration tasks, including super-resolution, deblurring, inpainting, and colorization, as well as more challenging applications such as low-light enhancement, HDR recovery, and LDR video enhancement. Experimental results demonstrate that GDP outperforms leading unsupervised methods across diverse benchmarks in both reconstruction accuracy and perceptual quality, while demonstrating robust generalization to images and videos of any size. Our project page at https://generativediffusionprior.github.io/.