Agentic AutoRAG: RAG Pipeline Optimization through Reasoning-Driven Agents

NLP RAG Agent Agent Planning Autonomous Workflows ML AutoML
86
2026年10月06日
检索增强生成（RAG）是一种广泛应用的技术，用于将大语言模型（LLM）锚定于外部知识。然而，构建一个高效RAG流水线本身是一项开销高昂的超参数优化任务——需在大量相互耦合的组件选择之间进行权衡，涵盖文本分块策略、嵌入模型、重排序方法以及生成模型等各个环节。当前主流优化方法（从贪心搜索到贝叶斯优化）均将每次试验简化为单一聚合评分，并在此基础上开展搜索；它们并未建模分析“为何某一配置表现不佳”，尽管每次检索所返回的文本片段本身已明确提供了关键线索：失败究竟源于检索阶段（即未召回相关证据），还是发生在后续生成阶段（即虽召回相关内容但未能正确推理作答）。为此，我们提出Agentic AutoRAG——一种面向多目标RAG超参数优化的LLM智能体优化器，其核心创新在于支持对“检索失败”与“生成失败”的归因诊断。该系统首先基于语料库构建一份固定不变的评测试卷，用于对候选配置进行打分；每次试验后，内置的“诊断器”（Diagnoser）会针对每一道答错的问题，精准判定其根本原因属于检索失败还是生成失败；随后，“提议器”（Proposer）依托一个整合了各模型性能排名与计费信息的知识库，综合权衡准确率与成本，自主决策下一轮待测配置，从而逐步追踪出准确率与成本之间的帕累托前沿（Pareto frontier）。在三个多跳问答基准测试中，Agentic AutoRAG所取得的LLM裁判评分准确率全面超越所有对比基线；尤为突出的是，在仅执行前10次试验的情况下，其准确率即已持平或反超各统计型基线经完整30次试验后达到的最终准确率。在面向真实世界医疗健康语料库的成本感知模式下，Agentic AutoRAG实现了77%的评测试卷中位准确率，显著高于最强基线的71.5%，同时单次查询成本仅为该基线的约58%；更进一步，当将其准确率目标设定为与基线持平（即71.5%）时，其单次查询成本可进一步压缩至基线水平的约22%。
Retrieval-augmented generation (RAG) is a widely used approach for grounding large language models (LLMs) in external knowledge. However, configuring a pipeline is an expensive hyperparameter optimization problem over many interacting choices, from chunking and embedding model to reranking and generation. Existing optimizers, from greedy search to Bayesian optimization, reduce each trial to an aggregate score and search without modeling why a configuration performed as it did, even though the retrieved chunks already provide evidence about whether each failure occurred during retrieval or after it. We introduce Agentic AutoRAG, an LLM-agent optimizer for multi-objective RAG hyperparameter optimization with retrieval-versus-generation failure attribution. It proposes configurations scored on a frozen exam from the corpus: after each trial a Diagnoser attributes each failed question to retrieval or generation, and a Proposer, grounded in a knowledge base of model rankings and pricing, selects the next configuration, weighing accuracy against cost to trace a Pareto frontier. On three multi-hop QA benchmarks it reaches higher LLM-judge accuracy than every baseline we compare, and within its first 10 trials it matches or beats the statistical baselines' full 30-trial judge accuracy. In its cost-aware mode on a real-world healthcare corpus it reaches a median exam accuracy of 77%, above the strongest baseline's 71.5%, at about 58% of that baseline's cost per query, and it matches that 71.5% at about 22% of the cost.
许愿