MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing

AK's Picks GenAI Diffusion TDIG SCCE
2026年07月06日
近期视频扩散模型的研究进展,要么通过时间维度上的自回归实现长时单视角视频生成,要么借助双向注意力机制实现短时多视角视频合成。然而,针对动态场景生成兼具长时序与多视角一致性的视频,这一难题至今仍未解决。本文提出一种名为 MV-Forcing 的新框架:它在单一扩散模型中协同整合时间维度与视角维度的自回归建模,并通过引入一种连接连续生成视角的四维几何桥梁(4D geometric bridge)来实现这一目标。我们的核心洞见在于:一个自回归式三维重建模型天然可作为各视角自回归生成过程之间的接口。具体而言,在完成源视角视频生成后,我们首先对其重建三维结构,再据此渲染出下一目标视角的几何先验(geometric prior),随后由扩散模型对该先验进行精细化处理,最终生成高质量视频。为突破教师模型(teacher model)所限定的固定时间窗口限制,我们设计了一种联合去噪训练范式(joint denoising regime):在训练阶段,所有视角插槽(view slots)均从噪声初始化,从而支持任意长度的时间序列生成。此外,我们采用“时空自强化分布匹配蒸馏”(Distribution Matching Distillation with Spatio-Temporal Self-Forcing)策略对模型进行知识蒸馏,有效弥合了训练与推理阶段在时间维度和视角维度自回归过程中存在的曝光偏差(exposure bias)鸿沟。在合成数据与真实世界数据上的大量实验表明,MV-Forcing 仅需一个轻量级、少步数(few-step)的学生模型,即可生成任意长度、任意视角数量的、具备几何一致性的动态场景多视角视频。
Recent advances in video diffusion models have enabled either long single-view generation through temporal autoregression, or short multi-view synthesis through bidirectional attention. However, generating long, multi-view consistent videos of dynamic scenes remains unsolved. In this work, we present MV-Forcing, a framework that composes temporal and view-wise autoregression within a single diffusion model by introducing a 4D geometric bridge between sequentially generated views. Our key insight is that an autoregressive 3D reconstruction model naturally interfaces between autoregressively generated views. Given a completed source view, we reconstruct its 3D structure and render a geometric prior of the next target viewpoint, which the diffusion model refines into a high-quality video. To extend generation beyond the teacher's fixed temporal window, we introduce a joint denoising regime where both view slots are initialized from noise during training, enabling temporally unbounded generation. We distill the model via Distribution Matching Distillation with Spatio-Temporal Self-Forcing, closing the train-inference exposure bias gap for both temporal and view-sequential autoregression. Extensive experiments on both synthetic and real-world data demonstrate that MV-Forcing produces geometrically consistent multi-view videos of dynamic scenes at arbitrary lengths and viewpoint counts using a single few-step student model.
许愿