VISTA: A Visual Harness for Reasoning in an Interactive World

Multimodal Intelligence Visual CoT ZSVR Agent Agent Planning Agent Memory
我们证明,多模态模型具备强大的推理能力,而恰当的框架设计能够充分释放其潜力,使其胜任各类交互式环境中的任务。为此,我们提出了VISTA——一种面向视觉任务的通用型评测框架,赋予通用多模态模型长时程视觉感知能力。VISTA使模型能够直接通过视觉观测感知环境,并维持一种无损的视觉记忆机制,将历史观测以原始形态完整保存。在推理过程中,模型可主动检索这些历史观测,并动态重组其视觉输入。在ARC-AGI-3基准测试中,VISTA将Claude Opus 5.0模型的“相对人类操作效率”(Relative Human Action Efficiency)得分从40.68大幅提升至满分100.00;该模型成功完成了全部25个公开测试关卡,且所用操作步数比首次参与的人类受试者减少了57.4%。VISTA结构简洁,因而能以极小的适配成本自然拓展至各类视觉环境。在另外三个涵盖多种视觉游戏与谜题的基准测试中,VISTA在采用相同底层模型的前提下,仅需极简的框架支持,即显著超越各类基线方法。我们的实验结果凸显了VISTA作为通用型视觉框架的巨大潜力,有望有力推动多模态智能体在复杂视觉环境中的能力发展。
We show that multimodal models possess strong reasoning abilities and that an appropriate harness can unlock their potential to solve tasks across diverse interactive environments. We introduce VISTA, a visual harness that gives a general-purpose multimodal model long-horizon vision. VISTA allows the model to directly perceive the environment through visual observations and maintains a lossless visual memory that preserves past observations in their original form. The model can actively retrieve these observations and reorganize its visual input as it reasons. On ARC-AGI-3, VISTA improves Claude Opus 5.0's Relative Human Action Efficiency score from 40.68 to a perfect 100.00, with the model completing all 25 public games using 57.4% fewer actions than first-time human participants. VISTA's simple design also allows it to extend naturally to diverse visual environments with minimal adaptation. Across three additional benchmarks covering a diverse range of visual games and puzzles, it substantially outperforms baselines using the same underlying model with minimal harnesses. Our results highlight VISTA's potential as a general-purpose visual harness for advancing multimodal agents in complex visual environments.
许愿