Long-WAM: Scaling the Context of World-Action Models

Wei Huang ,

Bohan Zhang ,

Chenzhi Liu ,

Isabella Liu ,

Shuai Yang ,

Weian Mao ,

Luozhou Wang ,

Yicheng Xiao ,

Weifeng Lin ,

Qixin Hu ,

Bryan Chu ,

Sifei Liu ,

Linxi Fan ,

Xiaojuan Qi ,

Song Han ,

Yukang Chen

AK's Picks Embodied AI and Robotics World Models RC

2026年10月07日

实时机器人控制需要足够长的视觉历史信息，以推断物体运动状态与任务执行进展；然而，处理这些历史信息本身又可能造成动作延迟。为此，我们提出了Long-WAM——一种面向实时控制约束、可扩展因果型“世界—动作”模型上下文长度的模型—系统联合框架。我们的核心发现是：拥有历史信息，不等于有效利用了该信息；当视频基础模型采用自回归（AR）方式预训练时，更长的历史上下文才能显著提升性能。我们首先在无动作标签的机器人操作视频与第一人称视角视频上学习因果预测能力，随后在将模型适配至“世界—动作”任务的过程中，完整保留这一“历史→未来”的建模结构。在RoboCasa GR-1基准上，将上下文长度从0.0秒增至19.2秒，任务成功率由63.3%提升至78.7%；而若采用双向（bidirectional）预训练初始化，则未观察到任何净收益；进一步采用面向机器人领域专门设计的自回归预训练策略，还可进一步提升GR-1与LIBERO-Long基准上的峰值成功率。Long-WAM在LIBERO-Long、RoboTwin 2.0及DOMINO三项基准测试中，亦均取得所有对比方法中的最优性能。通过流式观测编码、异步执行机制以及针对硬件平台定制的加速优化，Long-WAM可在RTX 5090、DGX Spark与Jetson AGX Thor等不同计算平台上实现零丢帧的实时部署；其中，在RTX 5090上，每个动作片段（含对未来视频隐空间表征的预测）的端到端耗时仅为107.4毫秒。在Unitree G1与YAM双足/轮式机器人平台上的实时部署结果表明，Long-WAM可稳健支持动态交互与长时程操作任务——例如在动态杯子叠放任务中达成95%的成功率，而Pi0.5与Fast-WAM在20次试验中全部失败。此外，作为具备记忆感知能力的底层执行器，Long-WAM亦能自然地与高层规划模块协同工作，共同完成复合型多阶段任务。

Real-time robot control demands enough visual history to infer motion and task progress, but processing that history can delay action. We present Long-WAM, a model-system framework for scaling the context of causal world-action models under real-time control constraints. Our central finding is that access to history is not the same as using it: longer histories pay off far more when the video foundation is pretrained autoregressively (AR). We first learn causal prediction from robot and egocentric videos without action labels, then preserve this history-to-future structure during world-action adaptation. On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, whereas a bidirectionally pretrained initialization shows no net gain; robot-domain AR pretraining further raises peak success on GR-1 and LIBERO-Long. Long-WAM also achieves the best results among compared methods on LIBERO-Long, RoboTwin 2.0, and DOMINO. Streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor without dropping future prediction; on RTX 5090, each action chunk, including future-video latent prediction, takes 107.4 ms. Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking, where Pi0.5 and Fast-WAM succeed in none of 20 trials. As a memory-informed executor, Long-WAM also complements higher-level planning in composite tasks.

许愿