Combating Missed Recalls in E-commerce Search: A CoT-Prompting Testing Approach

电子商务应用程序中的搜索组件通常是基于复杂的人工智能系统，容易出现错误，导致搜索结果中应该列出但未列出的商品，这种情况被称为“漏检”。这可能会让商家感到沮丧，并损害应用程序的盈利能力。然而，由于难以生成与用户需求相符的测试用例以及缺乏“神谕”，因此测试漏检是具有挑战性的。本文介绍了mrDetector，这是第一个专门针对漏检的自动化测试方法。为了解决测试用例生成的挑战，我们利用用户在搜索过程中构建查询的发现，通过LLM创建一个CoT提示来生成与用户需求相符的查询。此外，我们从创建多个查询并比较搜索结果的用户中学习，并通过变形关系提供测试神谕。使用开放访问数据的广泛实验表明，mrDetector优于所有基线，具有最低的假阳性比率。使用真实工业数据的实验表明，mrDetector只有17个假阳性，发现了超过100个漏检。

mrDetector: Automatic Testing for Missed Recalls in E-commerce Search Components

The paper proposes an automatic testing approach, mrDetector, specifically for missed recalls in e-commerce search components. It uses CoT prompts to generate user-aligned queries and provides a test oracle through a metamorphic relation.

mrDetector outperforms all baselines with the lowest false positive ratio. Experiments with real industrial data show that mrDetector discovers over one hundred missed recalls with only 17 false positives. The approach uses findings from how users construct queries during searching and learns from users who create multiple queries for one shop and compare search results.

Related work includes research on testing search components, generating test cases, and metamorphic testing. Some relevant papers are 'A Test Suite for Evaluating Search Engines' and 'Metamorphic Testing: A Review and New Perspectives'.