An Exam for Active Observers

AK's Picks LLM Other LLM Multimodal Intelligence Other MMI Agent Agent Eval Benchmarks
2026年07月17日
人类视觉是一个闭环系统:视线并非仅基于单一静态画面,而是持续依据中间假设进行动态调整。数十年来的心理物理学与认知科学研究均指出,这种主动式观察对完成大量任务至关重要。而当今的多模态大语言模型(MLLM)是否具备主动式观察能力,尚属一个有待实证检验的问题——现有的视觉-语言评测基准对此尚无答案。为此,我们提出“ActiveVision”基准,首次为MLLM的主动式观察能力提供可量化的评测框架,涵盖三大类共17项任务。这些任务经过专门设计,旨在强制模型进行多次、迭代式的视觉感知,而非仅依赖单次静态图像描述。当前最先进的MLLM在ActiveVision上表现极差:我们在评测中得分最高的模型——GPT-5.5(在其公开披露的最高推理努力层级下),仅能正确解答全部题目的10.6%,且在17项任务中的11项上得分为零;即便是近期在多数推理与编程排行榜上位居榜首的Claude Fable 5,其正确率也仅为3.5%,远低于三位人类受试者96.1%的平均正确率。此外,即使让模型自行编写并运行视觉处理代码,这一性能差距依然显著:此类自生成代码在真实场景图像上鲁棒性极低,而识别和修正其错误本身,恰恰又需要模型所缺乏的主动式观察能力。综上所述,这些结果表明,当前的MLLM普遍缺乏稳健可靠的主动式视觉观察能力,亟需构建新型架构与训练目标,以切实打通“感知—推理”的闭环。
Human vision is a closed loop: gaze is continuously redirected by intermediate hypotheses rather than a single snapshot. Decades of psychophysics and cognitive science have argued that this active observation is essential for a wide range of tasks. Whether today's multimodal large language models (MLLMs) exercise active observation is an empirical question that current vision-language benchmarks do not answer. We introduce ActiveVision, a benchmark that makes active observation measurable for MLLMs, comprising 17 tasks across 3 categories. Tasks are designed to force repeated visual perception rather than a single static description. Frontier MLLMs collapse on ActiveVision: the highest-scoring model we evaluate, GPT-5.5 at the highest exposed reasoning-effort tier, solves only 10.6% of items and scores zero on 11 of the 17 tasks, and even Claude Fable 5, despite topping most reasoning and coding leaderboards, solves just 3.5%, far behind three human participants who average 96.1%. Furthermore, much of the gap persists even when models write and run their own vision code: such code is unreliable on realistic imagery, and catching its failures itself requires the active perception the models lack. Together, these results indicate that current MLLMs lack robust active visual observation, motivating architectures and training objectives that close the perception-reasoning loop.
许愿