Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking

标准的自然语言处理（NLP）全数据分类器需要数千个标记的样本，这在数据有限的领域是不切实际的。少样本方法提供了一种替代方案，利用对比学习技术，每类只需要20个示例即可有效。同样，像GPT-4这样的大型语言模型只需要每类1-5个示例即可有效执行。然而，这些方法的性能成本权衡仍未得到充分探索，这是预算有限组织的关键问题。我们的工作通过在Banking77金融意图检测数据集上研究上述方法来填补这一空白，包括在全面的少样本场景下评估OpenAI、Cohere和Anthropic的尖端LLM。我们通过两种额外的方法来完善这个图景：首先，基于检索增强生成（RAG）的LLM成本有效的查询方法，能够将操作成本与经典的少样本方法相比降低多次；其次，使用GPT-4的数据增强方法，能够提高数据有限情况下的性能。最后，为了激发未来的研究，我们提供了一个人类专家精选的Banking77子集，以及广泛的错误分析。

The paper aims to explore the performance-cost trade-offs of few-shot methods and large language models in data-limited domains, specifically in the context of financial intent detection using the Banking77 dataset.

The paper proposes a comprehensive evaluation of different few-shot and large language model approaches in the context of financial intent detection. It also introduces two new methods - a cost-effective querying method for LLMs based on retrieval-augmented generation (RAG) and a data augmentation method using GPT-4.

The paper evaluates the performance of different few-shot and large language model approaches on the Banking77 dataset and provides a human expert's curated subset of the dataset along with extensive error analysis. It also introduces two new methods - a cost-effective querying method for LLMs based on retrieval-augmented generation (RAG) and a data augmentation method using GPT-4. The experiments are designed to explore the performance-cost trade-offs of these methods in data-limited domains. The paper also compares its results with some related work in the field.

Some related work in the field includes 'Few-shot learning for natural language processing: A survey' by Zhang et al., 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' by Raffel et al., and 'The Curious Case of Neural Text Degeneration' by Holtzman et al.