Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation

Haozhe Wang ,

Weijia Feng ,

Jinpeng Yu ,

Che Liu ,

Ping Nie ,

Fangzhen Lin ,

Jiaming Liu ,

Ruihua Huang ,

Jimmy Lin ,

Wenhu Chen ,

Cong Wei

AK's Picks Agent Tool Learning Agent Eval Benchmarks GenAI TDIG

2026年07月06日

视觉生成模型虽擅长图像渲染，却会对其未知内容自信地进行虚构。用户请求具有无限性、动态演化性与高度长尾性：涵盖全新角色、热门实体、训练截止日期之后发生的事件等各类需求。这一世界知识瓶颈本质上是结构性的：生成模型仅在固定语料库上进行训练，而视觉世界本身却是开放且持续演化的。为此，我们构建了SearchGen-20K与SearchGen-Bench两个基准数据集，共包含20,839条提示语，覆盖十二类典型失败场景及二十二个不同领域；同时配套提供一个已预先执行完毕的多模态语料库SearchGen-Corpus-1M，以支持离线、可复现的研究工作。在SearchGen-Bench上，当前最前沿的开源视觉生成模型得分仅为21至28分（满分100分），较其在现有基准上的表现出现高达40分的断崖式下滑——而这一严重退化现象在既有评测体系中完全无法察觉。直观的应对方案是引入搜索工具，从而实现具备自主能力（agentic）的视觉生成。然而，我们发现简单粗暴地直接调用搜索功能反而失效：它 indiscriminately（不加甄别地）检索信息，将大量噪声注入原本生成器本就能正确处理的提示语中。我们追溯根本原因，发现其在于生成器自身特有且随时间演化的知识边界——即模型通过训练所能内化掌握的知识，与必须依赖外部上下文才能获取的知识之间所存在的分界线。尽管该边界难以事先明确定义，但我们证明，可通过“先教学、后搜索”的协同训练框架予以有效识别。即使仅采用该协同训练范式的最简版本，亦能带来单调递进的性能提升，从而为视觉生成系统建立一条通向递归式自我增强的可行路径，使其真正满足基于真实世界知识的生成请求。我们已全面开源该数据集、协同训练语料库以及搜索语料库，构建起一套可回放、可复用的评估与开发平台，旨在推动工具增强型、世界知识驱动的视觉生成研究与实践。

Visual generators excel at rendering, but they confidently fabricate what they do not know. User requests are unbounded, evolving, and deeply long-tailed: new characters, trending entities, post-cutoff events, and more. This world-knowledge bottleneck is structural: generators are trained on fixed corpora, but the visual world is open-ended. We construct SearchGen-20K and SearchGen-Bench, with 20,839 prompts spanning twelve failure categories and twenty-two domains, paired with a pre-executed multimodal SearchGen-Corpus-1M to support offline, reproducible research. On SearchGen-Bench, frontier open generators score only 21 to 28 out of 100, a 40-point collapse invisible to existing benchmarks. The natural remedy is to employ search tools, enabling agentic visual generation. However, we find that naive search fails: it retrieves indiscriminately, injecting noise into prompts the generator already handles. We trace the root cause to a generator-specific, evolving knowledge boundary: the divide between what a generator can internalize through training and what must remain in external context. Although this boundary is hard to specify in advance, we show that it is discoverable through a teach-then-search co-training framework. Even a minimal version of this co-training recipe produces monotonic improvement, laying the foundation for recursive self-improvement in visual generation that can meet world-knowledge-grounded requests. We release the full dataset, co-training corpus, and search corpus as a replayable harness for tool-augmented, world-knowledge-grounded visual generation.

许愿