Revisiting Text Ranking in Deep Research

RecSys / IR DRSA Neural Search
深度研究已成为一项重要任务,旨在通过大规模开放网络探索来应对复杂查询。为解决该任务,现有主流方法通常为基于大语言模型(LLM)的智能体配备“黑盒式”网络搜索API,使其能够迭代地发起搜索请求、获取外部证据,并基于所获信息进行推理。尽管搜索在深度研究中扮演着核心角色,但此类不透明的网络搜索API却严重阻碍了对搜索各组成部分的系统性分析,致使当前主流文本排序方法在深度研究场景下的实际行为与表现仍缺乏清晰认知。为弥补这一空白,本文在深度研究设定下,复现并验证了一系列关键的检索(IR)文本排序方法的重要发现与最佳实践。具体而言,我们从以下三个维度考察其有效性:(i)检索单元(文档级 vs. 段落级);(ii)检索流程配置(不同检索器、重排序器及其重排序深度);(iii)查询特征(智能体生成的查询与文本排序模型训练时所用查询之间存在的语义与形式错配)。我们在BrowseComp-Plus数据集上开展实验——这是一个具有固定语料库的深度研究基准数据集,涵盖2个开源智能体、5种检索器与3种重排序器,并在多种实验配置下进行全面评估。实验结果表明:智能体生成的查询通常采用网页搜索风格的语法结构(例如带引号的精确匹配),因而更适配词法检索器、学习型稀疏检索器以及多向量检索器;在上下文窗口受限的情况下,段落级检索单元更具效率,且可规避词法检索中因文档长度差异带来的归一化难题;重排序环节效果极为显著;而将智能体生成的查询显式转化为自然语言问句,则能显著弥合前述查询错配问题。
Deep research has emerged as an important task that aims to address hard queries through extensive open-web exploration. To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it. Despite search's essential role in deep research, black-box web search APIs hinder systematic analysis of search components, leaving the behaviour of established text ranking methods in deep research largely unclear. To fill this gap, we reproduce a selection of key findings and best practices for IR text ranking methods in the deep research setting. In particular, we examine their effectiveness from three perspectives: (i) retrieval units (documents vs. passages), (ii) pipeline configurations (different retrievers, re-rankers, and re-ranking depths), and (iii) query characteristics (the mismatch between agent-issued queries and the training queries of text rankers). We perform experiments on BrowseComp-Plus, a deep research dataset with a fixed corpus, evaluating 2 open-source agents, 5 retrievers, and 3 re-rankers across diverse setups. We find that agent-issued queries typically follow web-search-style syntax (e.g., quoted exact matches), favouring lexical, learned sparse, and multi-vector retrievers; passage-level units are more efficient under limited context windows, and avoid the difficulties of document length normalisation in lexical retrieval; re-ranking is highly effective; translating agent-issued queries into natural-language questions significantly bridges the query mismatch.
许愿