基于域可泛化扩散模型的图像去模糊

来自今天的爱可可AI前沿推介。

[CV] Image Deblurring with Domain Generalizable Diffusion Models

M Ren, M Delbracio, H Talebi, G Gerig, P Milanfar
[Google Research & New York University]

基于域可泛化扩散模型的图像去模糊

简介：提出将图像条件扩散概率模型(icDPM)作为图像去模糊的有效工具，用含噪合成对进行训练，并以输入图像的域可泛化多尺度表示为指导，在三个外部及多样测试集上进行测试，以提高分布外性能。与现有方法相比，该方法具有最先进的性能。

摘要：扩散概率模型(DPM)最近被用于图像去模糊。DPM通过随机去噪过程训练而来，该过程将高斯噪声映射到高质量图像上，以串接模糊输入为条件。尽管其生成了高质量样本，但图像条件扩散概率模型(icDPM)依赖于合成的成对训练数据(域内)，对现实世界未见过的图像(域外)的鲁棒性可能不明确。本文研究了icDPM在去模糊中的泛化能力，并提出了一个简单而有效的指导，以显著减轻伪影，并提高域外性能。本文建议首先从输入图像中提取一个多尺度域可泛化表征，该表征在去除域特定信息的同时保留了图像的基本结构。该表征被添加到条件扩散模型的特征图，作为额外指导，帮助提高泛化能力。为了确定基准，将单一数据集训练的模型应用于三个外部及多样的测试集，重点关注分布外的性能。与现有方法相比，通过对标准icDPM的改进，以及在感知质量和竞争性失真指标上的最先进的表现，证明了所提方案的有效性。

论文链接：https://arxiv.org/abs/2212.01789

Diffusion Probabilistic Models (DPMs) have recently been employed for image deblurring. DPMs are trained via a stochastic denoising process that maps Gaussian noise to the high-quality image, conditioned on the concatenated blurry input. Despite their high-quality generated samples, image-conditioned Diffusion Probabilistic Models (icDPM) rely on synthetic pairwise training data (in-domain), with potentially unclear robustness towards real-world unseen images (out-of-domain). In this work, we investigate the generalization ability of icDPMs in deblurring, and propose a simple but effective guidance to significantly alleviate artifacts, and improve the out-of-distribution performance. Particularly, we propose to first extract a multiscale domain-generalizable representation from the input image that removes domain-specific information while preserving the underlying image structure. The representation is then added into the feature maps of the conditional diffusion model as an extra guidance that helps improving the generalization. To benchmark, we focus on out-of-distribution performance by applying a single-dataset trained model to three external and diverse test sets. The effectiveness of the proposed formulation is demonstrated by improvements over the standard icDPM, as well as state-of-the-art performance on perceptual quality and competitive distortion metrics compared to existing methods.

内容中包含的图片若涉及版权问题，请及时与我们联系删除