ParseBench: A Document Parsing Benchmark for AI Agents

NLP Benchmarks Agent Agent Eval Benchmarks
133
2026年04月09日
人工智能代理正在改变文档解析的需求。关键在于语义正确性：解析结果必须完整保留支撑自主决策所需的结构与语义信息，包括准确的表格结构、精确的图表数据、具有语义意义的格式化信息，以及与视觉内容的严格对齐（即“视觉锚定”）。现有基准测试未能全面反映企业自动化场景下的这一需求：它们所依赖的文档分布范围过窄，且仅采用文本相似度指标进行评估，因而无法有效识别对AI代理至关重要的各类失败情形。为此，我们提出ParseBench——一个面向企业级文档的新型基准测试集，涵盖保险、金融与政府三大领域，共包含约2000页经人工严格校验的文档页面；该基准围绕五大核心能力维度构建：表格解析、图表解析、内容保真度、语义化格式识别以及视觉锚定。在涵盖14种方法（包括多模态大模型、专用文档解析器及LlamaParse）的系统性评测中，ParseBench揭示出当前技术能力呈现显著碎片化特征：尚无任何一种方法能在全部五个维度上持续保持优异表现。其中，LlamaParse Agentic以84.9%的综合得分位居榜首；同时，该基准也清晰凸显了现有各类系统在关键能力上仍存在的明显短板。本数据集及配套评测代码已开源，可分别通过以下链接获取：https://huggingface.co/datasets/llamaindex/ParseBench 和 https://github.com/run-llama/ParseBench。
AI agents are changing the requirements for document parsing. What matters is semantic correctness: parsed output must preserve the structure and meaning needed for autonomous decisions, including correct table structure, precise chart data, semantically meaningful formatting, and visual grounding. Existing benchmarks do not fully capture this setting for enterprise automation, relying on narrow document distributions and text-similarity metrics that miss agent-critical failures. We introduce ParseBench, a benchmark of ${\sim}2{,}000$ human-verified pages from enterprise documents spanning insurance, finance, and government, organized around five capability dimensions: tables, charts, content faithfulness, semantic formatting, and visual grounding. Across 14 methods spanning vision-language models, specialized document parsers, and LlamaParse, the benchmark reveals a fragmented capability landscape: no method is consistently strong across all five dimensions. LlamaParse Agentic achieves the highest overall score at 84.9%, and the benchmark highlights the remaining capability gaps across current systems. Dataset and evaluation code are available on https://huggingface.co/datasets/llamaindex/ParseBench and https://github.com/run-llama/ParseBench.
许愿