Principia: Relational Physics Tests for Video Models

AK's Picks Multimodal Intelligence VideoQA Other MMI GenAI TDIG SCCE
2026年09月03日
在视频模型中评估物理推理能力十分困难,因为绝对运动的测量结果依赖于帧率、物体尺度以及相机标定参数——而这些信息在生成的视频中往往模糊不清,甚至完全不可获取。为此,我们提出一种不同的评估思路:当同一场景中的两个物体遵循相同的物理规律时,它们的运动之间必然存在可预测的关系;而这类关系的成立并不依赖于任何相机或场景的标定信息。我们由此构建了评测基准“Principia”,它通过成对物体之间的关系一致性来评估模型对牛顿力学原理的掌握程度。“Principia”覆盖八大物理现象——重力、恢复系数(碰撞反弹)、摩擦力、转动惯量、抛体运动、动量守恒、单摆运动以及质量-弹簧振荡系统——并涵盖平动、转动、碰撞及振荡等各类动力学过程,所有测试视频均采用受控实验协议采集的真实世界场景。此外,我们还设计了一种无需标定的“一致性得分”,该指标直接在图像空间内量化物理规律被违背的程度。在对六种当前最先进的视频生成模型所产出的数千段视频进行测试后,没有任何一个模型在 Principia 上得分超过 0.42,尽管它们在 VBench 基准上的平均得分均接近 0.8。我们还进一步评估了多模态视觉-语言模型识别此类关系性物理错误的能力:表现最优的模型准确率仅为 67%,而大多数模型的表现则接近随机猜测水平。
Evaluating physical reasoning in video models is difficult because absolute motion measurements depend on frame rate, object scale, and camera calibration, all of which are often ambiguous or unavailable in generated video. We propose a different approach. When two objects in the same scene obey the same physical law, their motions must satisfy predictable relationships, and these relationships hold independent of calibration. We introduce Principia, a benchmark that evaluates Newtonian physics through relational consistency between paired objects. Principia spans eight phenomena - gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation - across translational, rotational, collisional, and oscillatory dynamics, using real-world scenes recorded under controlled protocols. We also introduce a calibration-independent consistency score that quantifies physical violation directly in image space. Across thousands of generations from six state-of-the-art video generators, no model exceeds 0.42 on Principia despite all scoring around 0.8 on VBench. Vision-language models are evaluated on their ability to detect relational physics violations, with the best model achieving only 67% accuracy and most performing near chance level.
许愿