Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success

Embodied AI and Robotics MLMRD Multimodal Intelligence FGCMA
2026年10月22日
机器人基础模型通常需通过成本高昂的闭环 rollout(即在真实或仿真环境中执行完整任务流程)进行评估,因此亟需一种低成本的离线代理指标来衡量模型能力。柏拉图表征假说(Platonic Representation Hypothesis, PRH)指出:随着表征能力不断提升,不同模态(如视觉与语言)将逐渐收敛于同一套对现实世界的统计建模。我们进一步追问:运动轨迹是否构成另一类具备同等地位的模态?换言之,当具身模型的能力持续增强时,其感知表征是否也应趋向于物理运动本身的结构?为验证这一问题,我们提出“感知-运动对齐度”(Sensorimotor Alignment,简记为 $S^2$)——一种无需训练、完全离线的评估指标,用于量化模型感知表征与物理运动几何结构之间的匹配程度。实验发现,$S^2$ 在多种具身模型家族中均与下游机器人任务成功率呈显著正相关,因而可作为一种低成本的离线代理指标,用于在开展耗时耗力的闭环评估前,对候选模型进行优先级排序。此外,$S^2$ 甚至可在尚未对视觉-语言模型(VLM)进行昂贵的微调、将其转化为视觉-语言-动作模型(VLA)之前,便有效识别出具备潜力的VLM骨干网络。除相关性之外,$S^2$ 还能从表征层面解释视觉-语言-动作模型(VLA)及世界动作模型(WAM)中观察到的诸多现象。借助 $S^2$,我们将柏拉图表征假说(PRH)拓展至具身智能体领域,并首次实现了对感知-运动对齐度的实证可测量性,从而不仅揭示了具身模型“为何成功”,更超越了单纯依赖闭环任务成功率的传统评估范式,为理解模型内在能力提供了深层洞见。
Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ($S^2$), a training-free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$, we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.
许愿