Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

Embodied AI and Robotics World Models Agent Multi-agent Collaboration GenAI Diffusion TDIG
现有面向交互式视频生成的世界模型,主要集中于单智能体场景,即仅依据单一控制信号来生成未来观测结果。然而,许多需要生成的虚拟环境本质上是多智能体交互的:多个玩家、机器人或具身智能体需在同一共享空间中同步行动。将世界模型扩展至此类多智能体场景,亟需一套系统化、原理清晰的多智能体建模框架:各智能体须保持独立可控性,整体架构应对智能体排列顺序保持对称性(即置换不变性),同时兼顾高效推理能力,并在时间维度与视角维度上维持行为一致性。本文提出一种面向交互式仿真的生成式多智能体世界模型。该模型引入“单纯形旋转智能体编码”(Simplex Rotary Agent Encoding)——这是一种无需额外参数的三维旋转位置编码(3D RoPE)扩展方法,其将各智能体映射为旋转角空间中一个正则单纯形(regular simplex)的顶点。该设计既赋予每个智能体独特的相位特征,又保证所有智能体在数学上完全可互换(permutation-equivalent),从而在不依赖可学习的槽位专属标识(per-slot identities)或预设智能体固定排序的前提下,实现可扩展的智能体身份表征。为避免智能体间采用计算开销高昂的全连接注意力机制(all-to-all attention),我们进一步提出“稀疏中心注意力”(Sparse Hub Attention):通过引入若干可学习的中心令牌(hub tokens)作为跨智能体信息交互的中介,将智能体间注意力的计算复杂度由智能体数量的平方级(quadratic)降至线性级(linear)。针对实时视频推演需求,我们还采用知识蒸馏策略,将具备完整上下文感知能力的扩散模型教师网络(full-context diffusion teacher)蒸馏为因果式学生网络(causal student);该学生网络以时序块(temporal blocks)为单位逐块生成视频,并结合键值缓存(KV caching)机制,最终实现在24帧/秒(FPS)下的动作响应式视频生成。在多人虚拟环境中的实验表明,相较于基于槽位(slot-based)和全连接注意力(dense-attention)的基线方法,本模型在视频保真度、动作可控性以及智能体间行为一致性三方面均取得显著提升;更值得注意的是,模型在仅用双人数据训练的情况下,可无缝泛化至四人场景,无需任何额外训练。
World models for interactive video generation have largely focused on single-agent settings, where future observations are generated from a single control signal. However, many generated environments require multi-agent interaction: multiple players, robots, or embodied agents act simultaneously within a shared space. Scaling world models to such settings requires a principled multi-agent design: agents should remain independently controllable, permutation-symmetric, and support efficient inference while maintaining consistency across time and perspectives. In this paper, we present our generative multi-agent world model for interactive simulation. It introduces Simplex Rotary Agent Encoding, a parameter-free extension of 3D RoPE that represents agents as vertices of a regular simplex in rotary angle space. This gives each agent a distinct phase while making all agents permutation-equivalent, enabling scalable agent identity without learned per-slot identities or a fixed agent ordering. To avoid dense all-to-all attention across agents, we further propose Sparse Hub Attention, where learnable hub tokens mediate token interaction across agents, reducing cross-agent attention cost from quadratic to linear in the number of agents. For real-time rollout, we distill a full-context diffusion teacher into a causal student that generates temporal blocks sequentially with KV caching, enabling action-responsive generation at 24 FPS. Experiments in multiplayer virtual environments show that our model improves video fidelity, action controllability, and inter-agent consistency over slot-based and dense-attention baselines, while generalizing from two to four players without additional training.
许愿