Pictura: Perspective-View Self-Play at Scale for Driving

Embodied AI and Robotics AD Sim-Platform ML PGPO MARL
2026年07月28日
在仿真环境中进行自博弈训练，可大规模生成鲁棒的驾驶策略。此前相关行为的演示均依赖于具备“特权”的向量化观测信息（例如精确的位姿与速度），甚至对被遮挡的交通参与者亦是如此。这种设定隐含了一个前提：感知问题已被完全解决，从而在实际部署时造成了一种表征鸿沟——即训练所用的完整、精确状态信息，与真实车载智能体仅能通过前视/自车视角摄像头获取部分观测之间存在根本性差异。一种常见应对方案是将具备特权信息的教师策略蒸馏为以相机图像为输入的学生策略，但该方法导致学生策略所模仿的决策，往往超出了其自身视觉观察能力所能合理支撑的范围。为此，我们提出并确立了基于视角图像的自博弈训练范式，使之成为一种切实可行的训练方式。我们推出了Pictura——一款GPU加速的多智能体驾驶仿真器，其核心创新在于：在每个仿真步中，实时渲染每一智能体的自车视角图像，从而从源头上消解上述表征鸿沟。Pictura在单块英伟达H100 GPU上即可实现高达50万智能体步/秒（即200万张图像/秒）的吞吐量。依托Pictura平台，我们采用标准近端策略优化（PPO）算法，通过纯视角图像驱动的自博弈方式成功训练出Alberti模型。这是首个完全基于前视图像、不依赖任何特权观测信息的大规模驾驶自博弈策略。整个训练过程共涵盖500亿智能体步，对应约3500万公里的虚拟驾驶里程。该模型的驾驶性能已逼近同等条件下使用特权向量化观测所训练的基准模型；更值得注意的是，它还能零样本迁移至由Pictura重新渲染的Waymo开放运动数据集（Waymo Open Motion Dataset）场景中，并在该场景下全面超越基于特权向量化观测的同类智能体。项目主页：https://valeoai.github.io/Pictura/
Self-play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera-input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective-view self-play as a practical training regime. We introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent-steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self-play with plain PPO. It is the first large-scale driving self-play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero-shot to Waymo Open Motion Dataset layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/
许愿