Does Document Structure Help Dense Retrieval? A Placebo-Controlled Ablation of Four Mechanisms Across Two Corpora

NLP RAG DRRO RecSys / IR DRSA Neural Search
检索增强型生成系统正日益依赖于对文档结构的处理方法,包括:按结构对齐的文本分块(structure-aligned chunking)、由大语言模型(LLM)生成的分块上下文、标题路径元数据(heading-path metadata),以及分层级的两阶段检索。此前各项研究分别在不同语料库、不同嵌入模型(embedders)和不同评估指标下验证了上述每种方法的有效性,但均未控制一个共同的混杂因素:任何前置到文本分块之前的附加内容都会干扰该分块的嵌入表征(embedding)。为此,我们设计了一种机制隔离式的消融实验,在统一实验协议下系统评估全部四种结构处理方式;实验中严格匹配各条件下的分块长度,并额外引入一种语义上“无效”但结构上合法的安慰剂对照组——即从其他文档中随机打乱拼接而成的标题路径(shuffled heading paths)。我们采用一种兼顾覆盖率的归一化折损累积增益(coverage-aware nDCG)作为检索效果的评估指标,并基于文档簇的自助法(document-clustered bootstrap)结合霍尔姆校正(Holm correction),对预先注册的四组对比假设进行统计检验。实验在两个差异显著的语料库上开展:200 篇维基百科特色条目(含 951 个查询)与 1,585 篇 QASPER 学术论文(含 4,303 个问题)。结果表明,合理的文档组织确实有助于提升检索性能,且其作用根源在于内容本身,而非单纯增加的词元(tokens)数量:采用真实标题路径、按结构对齐生成的分块,其表现显著优于经上下文化处理的固定窗口分块(cov-nDCG@10 分别高出 +0.022 / +0.012)以及上述安慰剂对照组(+0.010 / +0.016)。相反,朴素的两阶段分层检索策略反而损害性能(-0.033 / -0.015),其原因可追溯至第一阶段对章节(section)的召回率偏低。在维基百科语料上,“黄金标准”(gold)人工标注的结构优于 LLM 自动生成的结构;但在 QASPER 语料上则未观察到这一优势。尽管效应量较小(Cohen’s dz = 0.06–0.11),但所有效应均通过霍尔姆校正后的显著性检验,且在两个语料库上结果高度一致。
Retrieval-augmented generation systems increasingly rely on document-structure treatments: structure-aligned chunking, LLM-generated chunk contexts, heading-path metadata, and hierarchical two-stage retrieval. Separate studies support each on different corpora, embedders, and metrics, and none control for a shared confound: any text prepended to a chunk perturbs its embedding. We present a mechanism-isolating ablation testing all four treatments under one protocol, matching chunk sizes across conditions and adding a semantically null placebo---heading paths that are structurally valid but shuffled across documents. We score retrieval with a coverage-aware nDCG and test four pre-registered contrasts via document-clustered bootstrap with Holm correction, on two distant corpora: 200 Wikipedia Featured Articles (951 queries) and 1,585 QASPER papers (4,303 questions). Organization helps, and the cause is content, not tokens: structure-aligned chunks with real heading paths beat contextualized fixed windows (+0.022 / +0.012 cov-nDCG@10) and the placebo (+0.010 / +0.016). Naive two-stage hierarchical retrieval hurts (-0.033 / -0.015), traceable to first-stage section recall. Gold structure beats LLM-induced structure on Wikipedia but not on QASPER. Effects are small ($dz$ 0.06-0.11) but Holm-significant and consistent across corpora.
许愿