来自今天的爱可可AI前沿推介

[IR] Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification

J Gupta, Y Tay, C Kamath, V Q. Tran, D Metzler, S Bavadekar, M Sun, E Gabrilovich
[Google Research]

用于新冠疫苗接种搜索分类的稠密特征记忆增强Transformer

要点:

  1. 提出一种新的搜索查询意图分类模型和框架，作为COVID-19疫苗相关搜索的洞察工具；
  2. 结合现代最先进的自然语言理解模型以及传统的稠密特征，提出一种新的融合方法，使查询能以类似上下文键值存储的方式从稠密记忆存储中检索查询；
  3. 实验证明，该方法可显著改善强大的梯度提升基线，且F1得分超越最先进的Transformer。

摘要：
随着新冠肺炎的全面爆发，疫苗是应对全球大流行中大规模感染的关键防线之一。鉴于其提供的保护，疫苗在某些社会和专业环境中成为强制性的。本文提出一种用于检测新冠肺炎疫苗接种相关搜索查询的分类模型，一种用于生成新冠肺炎疫苗接种搜索见解的机器学习模型。该方法结合并利用了现代最先进的(SOTA)自然语言理解(NLU)技术的进步，例如具有传统稠密特征的预训练Transformer。提出一种将稠密特征视为模型可处理记忆token的新方法。这种新的建模方法可以显著改进疫苗搜索洞察(VSI)任务，通过F1得分相对提高+15%和精度提高+14%来提高精心构造的梯度提升基线。

With the devastating outbreak of COVID-19, vaccines are one of the crucial lines of defense against mass infection in this global pandemic. Given the protection they provide, vaccines are becoming mandatory in certain social and professional settings. This paper presents a classification model for detecting COVID-19 vaccination related search queries, a machine learning model that is used to generate search insights for COVID-19 vaccinations. The proposed method combines and leverages advancements from modern state-of-the-art (SOTA) natural language understanding (NLU) techniques such as pretrained Transformers with traditional dense features. We propose a novel approach of considering dense features as memory tokens that the model can attend to. We show that this new modeling approach enables a significant improvement to the Vaccine Search Insights (VSI) task, improving a strong well-established gradient-boosting baseline by relative +15% improvement in F1 score and +14% in precision.

论文链接：https://arxiv.org/abs/2212.13898
图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除