WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory

AK's Picks Embodied AI and Robotics World Models GenAI SCCE
2026年07月02日
我们提出了WorldDirector——一种高度可控的视频世界模型框架,专为实现持久化的动态物体记忆与无限制的视角探索而设计。与现有世界模型不同(后者将物理动力学与像素级渲染耦合在一起,并依赖连续的视觉观测来维持运动),本框架明确地将语义层面的运动编排与视觉生成解耦。具体而言,我们利用大语言模型(LLM)协调三维运动轨迹与相机运动,并将这些协同规划好的轨迹作为控制信号输入至视频生成模块;该方法不仅严格遵循物理规律,还保障了外观的一致性与稳定性,即使动态物体在长时间离开视野后重新进入画面,其精确的视觉身份(如外观、纹理、形状等)仍能被完整保留。实验结果表明,本方法能够以前所未有的可控性,合成复杂且持续时间较长的事件,并具备强大的动态物体长期记忆能力。项目主页:https://worlddirector.github.io/
We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/
许愿