PnP-Flow: Plug-and-Play Image Restoration with Flow Matching

GenAI Diffusion Inpainting CV SR and Denoising
2024年10月03日
本文介绍了一种解决成像反问题的算法,称为Plug-and-Play(PnP)Flow Matching。PnP方法利用预先训练的去噪器(通常是深度神经网络),将它们整合到优化方案中。虽然它们在成像的各种反问题上实现了最先进的性能,但PnP方法在像修复这样更具生成性的任务上面临固有的局限性。另一方面,生成模型(如Flow Matching)推动了图像采样的边界,但缺乏一种清晰的方法来有效地用于图像恢复。我们建议将PnP框架与Flow Matching(FM)相结合,通过使用预训练的FM模型定义一个时间相关的去噪器。我们的算法在数据保真度项上交替进行梯度下降步骤、学习的FM路径上的投影和去噪。值得注意的是,我们的方法计算效率高,内存友好,因为它避免了对ODE和跟踪计算的反向传播。我们评估了它在去噪、超分辨率、去模糊和修复任务中的性能,展示了比现有的PnP算法和基于Flow Matching的最先进方法更优异的结果。
In this paper, we introduce Plug-and-Play (PnP) Flow Matching, an algorithm for solving imaging inverse problems. PnP methods leverage the strength of pre-trained denoisers, often deep neural networks, by integrating them in optimization schemes. While they achieve state-of-the-art performance on various inverse problems in imaging, PnP approaches face inherent limitations on more generative tasks like inpainting. On the other hand, generative models such as Flow Matching pushed the boundary in image sampling yet lack a clear method for efficient use in image restoration. We propose to combine the PnP framework with Flow Matching (FM) by defining a time-dependent denoiser using a pre-trained FM model. Our algorithm alternates between gradient descent steps on the data-fidelity term, reprojections onto the learned FM path, and denoising. Notably, our method is computationally efficient and memory-friendly, as it avoids backpropagation through ODEs and trace computations. We evaluate its performance on denoising, super-resolution, deblurring, and inpainting tasks, demonstrating superior results compared to existing PnP algorithms and Flow Matching based state-of-the-art methods.
许愿