Velocity Scaling in Flow Matching

GenAI Diffusion ML Generative Models
近期研究表明,对学习得到的流匹配速度场 $v_θ$ 按时间函数 $γ(t)$ 进行缩放,可显著提升生成质量。先前研究认为:采用均方误差(MSE)训练的速度场会系统性地低估速度幅值,而缩放操作正是为了校正这一偏差。我们指出,MSE 训练本身并不会导致速度幅值不足。相反,我们发现速度缩放的作用在于减小“种群时间滞后”(population time lag):即模型在时间 $t$ 采样得到的状态,其分布更接近训练数据中更早时刻(小于 $t$)对应的状态。换言之,速度缩放与将模型时间整体前移,是两种等效地缓解种群时间滞后的策略。我们在多种网络架构和不同模型规模上均验证了该现象:通过量化种群时间滞后并据此选取最优缩放增益 $γ(t)$,可大幅提高生成质量;在 ImageNet-256 数据集上、无分类器引导(no guidance)、步数(NFE)为 25 的设定下,FID 指标由 28.0 降至 12.2(该数值系依据邻近增益点测得的 FID 值经线性插值得到)。
Scaling a learned flow-matching velocity field $v_θ$ by a gain $γ(t)$ was recently shown to greatly improve generation quality. Prior work argued that velocity fields trained with mean-squared error (MSE) systematically underestimate velocity magnitude and that scaling corrects this error. We show that MSE training does not create a velocity-magnitude deficit. We find instead that velocity scaling reduces population time lag: sampled states at model time $t$ resemble training states from an earlier time. Velocity scaling and moving model time back are two ways to address this population time lag. Across architectures and model sizes, measuring population time lag and using it to select a gain greatly improves generation quality, reducing FID from 28.0 to 12.2 (estimated by linear interpolation between FID measurements at neighboring gains) on ImageNet-256 at NFE 25 without guidance.
许愿