$π\mathbf{R}^2$: Reactive Real-time Flow Policies

AK's Picks Embodied AI and Robotics Imitation Learning RC IL GenAI Diffusion
2026年07月28日
通用型操作策略正日益采用基于大规模预训练骨干网络构建的“动作分块流式策略”（action-chunking flow policies）。此类分块策略以开环方式运行，因此在执行过程中无法响应实时传入的感知输入，从而牺牲了策略的**反应性**（reactivity）。虽然更频繁地重新规划（replanning）可在理论上恢复反应性，但当前从感知到动作的完整处理流水线（包含一个大型骨干网络及多次去噪步骤）速度过慢：这种**延迟**（latency）限制了高频重规划能力，导致已提交的动作长期滞后、过时，使得这类策略难以胜任动态环境下的闭环控制任务。为此，我们提出 $π\mathbf{R}^2$——一种可在保留大型骨干网络、表达能力强的多模态策略以及多动作预测能力的前提下，显著提升策略反应性与实时性的新方法。该方法基于扩散强制（diffusion forcing）所采用的“逐位置噪声调度”（per-position noise schedule），主要贡献两项创新设计：第一，将条件输入（conditioning）拆分为两条通道——一条**快速通道**（fast channel），实时接入本体感觉信号（proprioception），每轮控制周期均刷新；另一条**异步更新的慢速通道**（asynchronously updated slow channel），承载视觉-语言特征，更新频率较低。由此，策略可在单个动作分块内部即时响应本体感觉变化，同时对视觉信息的滞后具有鲁棒性；第二，引入一种**延迟自适应的流式调度机制**（latency-adaptive flow schedule）：将正在执行中的动作视作“图像修复式”（inpainting）条件输入，并在每次调用时仅通过单次去噪步骤即输出新动作，从而使同一个训练好的模型能够灵活适配不同硬件平台所固有的延迟差异。$π\mathbf{R}^2$ 对现有架构改动极小，可直接从预训练策略出发进行微调。我们在真实 xArm6 + XHand 机器人平台上，将该方法应用于 GR00T-N1.7 模型，实现了约 **4 倍于基线策略的闭环重规划速度**（在 A5000 GPU 上达 ~25 Hz），即每 **40 毫秒**即可基于一次全新观测生成并执行动作。在涵盖仿真与真实世界操作任务的广泛评测中，$π\mathbf{R}^2$ 相较最强基线方法，任务成功率分别提升高达 **23%（仿真环境）** 与 **30%（真实世界）**。项目主页：https://pi-r2-flow.github.io/
Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot react to sensory input arriving mid-execution, sacrificing \emph{reactivity}. Replanning more often would restore it, but the perception-to-action pipeline (a large backbone plus multiple denoising steps) is too slow: this \emph{latency} forbids frequent replanning and leaves committed actions stale, making such policies ill-suited for dynamic, closed-loop control. We present $π\mathbf{R}^2$, which makes these policies reactive and real-time while retaining large backbones, expressive multi-modal policies, and multi-action prediction. Built on the per-position noise schedule of diffusion forcing, $π\mathbf{R}^2$ contributes two ideas. First, it splits conditioning into a fast channel (proprioception, fresh every tick) and an asynchronously updated slow channel (vision-language features), so the policy reacts to proprioception within a chunk while tolerating stale vision. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits actions in one denoising step per call, letting one trained model adapt to varying hardware latency. Requiring minimal modification to existing architectures, $π\mathbf{R}^2$ can be finetuned from a pretrained policy: applied to GR00T-N1.7 on a real xArm6+XHand platform, it replans closed-loop roughly $4\times$ faster than the base policy (~$25$Hz on an A5000 GPU), acting on a fresh observation every $40$ms. Across simulation and real-world manipulation tasks, $π\mathbf{R}^2$ improves the success rate by up to $23\%$ in simulation and $30\%$ in the real world over the strongest baseline. Project page: https://pi-r2-flow.github.io/
许愿