Hierarchical Denoising For Multi-Step Visual Reasoning

AK's Picks Embodied AI and Robotics Imitation Learning World Models Multimodal Intelligence Visual CoT ZSVR GenAI Diffusion TDIG
2026年07月16日
视频模型正逐步演进为视觉基础模型,但其在类人多步推理能力方面仍显不足。流式自回归扩散模型虽具备较高的推理效率,却受限于推理能力;而双向扩散模型虽能实现全局修订,但由于需对每一帧进行密集去噪,导致推理开销过高。上述两类范式均难以兼顾复杂推理任务所需的逻辑一致性与低延迟流式输出。为此,我们提出HDR(面向视觉推理的分层去噪框架),一种将分层隐变量整合至因果视频生成过程中的统一框架,以支持多步推理。HDR将视频隐变量组织为树状分层结构,从而在流式输出前实现由粗到细的渐进式推理:较粗粒度的去噪层保留不确定性假设,以支持全局规划;更细粒度的层则逐步将这些假设细化为具体的视觉状态。此外,我们设计了一种稀疏分层注意力模式(SHAP),进一步降低了时间维度上的注意力计算开销。我们还构建了一个按层级划分的多步视频推理基准测试集,涵盖分布外(out-of-distribution)场景,并包含六类任务:迷宫导航、汉诺塔、一笔画、滑块拼图、推箱子(Sokoban)以及倒水问题。相较于流式自回归扩散基线模型,HDR的成功率从34.22提升至60.29(相对提升76.2%),平均任务进展度亦从76.00提高至89.56,展现出更为一致、稳健的推理轨迹。HDR在保持低延迟流式生成的同时,单隐变量处理耗时仅为0.70秒,推理速度达双向扩散模型的54.2倍。在仅使用2%训练数据的情况下,HDR仍可保持全量数据训练性能的82.9%,而双向扩散模型对应指标仅为52.0%。真实世界机器人实验进一步验证了HDR在物理交互与世界建模方面的应用潜力。项目演示页面:https://hierarchical-diffusion-reasoning.github.io/。
Video models are evolving into vision foundation models, yet they still lack human-like multi-step reasoning. Streaming autoregressive diffusion models are efficient but limited in reasoning, while bidirectional diffusion enables global revision with high inference costs due to dense frame-level denoising. Both paradigms struggle to achieve logical consistency and low-latency streaming for complex reasoning tasks. We propose HDR (Hierarchical Denoising for Visual Reasoning), a unified framework that integrates hierarchical latents into causal video generation for multi-step reasoning. HDR organizes video latents into a tree-structured hierarchy, enabling coarse-to-fine reasoning before streaming output. Coarse denoising layers preserve uncertain hypotheses for global planning, while finer layers progressively refine them into concrete visual states. A sparse hierarchical attention pattern (SHAP) further reduces temporal attention costs. We introduce a level-stratified multi-step video reasoning benchmark with out-of-distribution cases, covering six tasks: maze navigation, Tower of Hanoi, one-line drawing, sliding puzzle, Sokoban, and water pouring. Compared with streaming autoregressive diffusion baselines, HDR improves success from 34.22 to 60.29 (76.2% relative gain) and increases average progress from 76.00 to 89.56, demonstrating more consistent reasoning trajectories. HDR maintains low-latency streaming at 0.70 seconds per latent, achieving 54.2 times faster inference than bidirectional diffusion. It also retains 82.9% of full-data performance with only 2% training data, compared with 52.0% for bidirectional diffusion. Real-world robot experiments further demonstrate HDR's potential for physical interaction and world modeling. Project demo: https://hierarchical-diffusion-reasoning.github.io/.
许愿