Are First-Order Diffusion Samplers Really Slower? A Fast Forward-Value Approach

GenAI Diffusion ML DM
高阶常微分方程(ODE)求解器已成为加速扩散概率模型(DPM)采样的标准工具,这使得人们普遍认为一阶方法本质上更慢,而提高离散化阶数是实现更快生成的主要途径。本文挑战了这一观点,并从一个互补的角度重新审视了加速机制:除了求解器的阶数之外,在反向时间动态过程中DPM评估点的布置方式在神经网络函数评估次数(NFE)较低的情况下会显著影响采样的准确性。我们提出了一种新颖的无需训练的一阶采样器,其主导离散误差的符号与DDIM相反。从算法上看,该方法通过一种廉价的单步前向预测器来近似前向值评估。我们提供了理论保证,证明所得到的采样器能够准确逼近理想的前向值轨迹,同时保持一阶收敛性。实验结果表明,在标准图像生成基准(CIFAR-10、ImageNet、FFHQ 和 LSUN)上,该采样器在相同的 NFE 预算下始终提升了样本质量,且性能可与最先进的高阶采样器相媲美,有时甚至更优。总体而言,这些结果表明,DPM 评估点的布置为加速扩散采样提供了一个额外且相对独立的设计维度。
Higher-order ODE solvers have become a standard tool for accelerating diffusion probabilistic model (DPM) sampling, motivating the widespread view that first-order methods are inherently slower and that increasing discretization order is the primary path to faster generation. This paper challenges this belief and revisits acceleration from a complementary angle: beyond solver order, the placement of DPM evaluations along the reverse-time dynamics can substantially affect sampling accuracy in the low-neural function evaluation (NFE) regime. We propose a novel training-free, first-order sampler whose leading discretization error has the opposite sign to that of DDIM. Algorithmically, the method approximates the forward-value evaluation via a cheap one-step lookahead predictor. We provide theoretical guarantees showing that the resulting sampler provably approximates the ideal forward-value trajectory while retaining first-order convergence. Empirically, across standard image generation benchmarks (CIFAR-10, ImageNet, FFHQ, and LSUN), the proposed sampler consistently improves sample quality under the same NFE budget and can be competitive with, and sometimes outperform, state-of-the-art higher-order samplers. Overall, the results suggest that the placement of DPM evaluations provides an additional and largely independent design angle for accelerating diffusion sampling.
许愿