ChatGPT用于推荐系统的初步研究

Is ChatGPT a Good Recommender? A Preliminary Study

Junling Liu, Chao Liu, Renjie Lv, Kang Zhou, Yan Zhang
[Alibaba Group]

要点:

动机：探索将ChatGPT用于推荐系统的潜力，研究它是否能够在推荐场景中有效地迁移大规模语言和世界知识。
方法：构建一组提示，评估ChatGPT在五种推荐场景下的性能表现，使用少样本提示注入交互信息以帮助ChatGPT更好地理解用户需求和偏好。
优势：发现ChatGPT在某些任务中表现出色，并超越了最先进的方法。

探讨了将ChatGPT用于推荐系统的潜力，设计了一组提示来评估ChatGPT在五种不同的推荐场景下的性能表现，包括评分预测、顺序推荐、直接推荐、解释生成和评论摘要。通过构建一个基准测试，发现ChatGPT在某些任务中表现出色，在其他任务中则表现平庸。

Recommendation systems have witnessed significant advancements and have been widely used over the past decades. However, most traditional recommendation methods are task-specific and therefore lack efficient generalization ability. Recently, the emergence of ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. Nonetheless, the application of ChatGPT in the recommendation domain has not been thoroughly investigated. In this paper, we employ ChatGPT as a general-purpose recommendation model to explore its potential for transferring extensive linguistic and world knowledge acquired from large-scale corpora to recommendation scenarios. Specifically, we design a set of prompts and evaluate ChatGPT’s performance on five recommendation scenarios, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. Unlike traditional recommendation methods, we do not fine-tune ChatGPT during the entire evaluation process, relying only on the prompts themselves to convert recommendation tasks into natural language tasks. Further, we explore the use of few-shot prompting to inject interaction information that contains user potential interest to help ChatGPT better understand user needs and interests. Comprehensive experimental results on Amazon Beauty dataset show that ChatGPT has achieved promising results in certain tasks and is capable of reaching the baseline level in others. We conduct human evaluations on two explainability-oriented tasks to more accurately evaluate the quality of contents generated by different models. And the human evaluations show ChatGPT can truly understand the provided information and generate clearer and more reasonable results. We hope that our study can inspire researchers to further explore the potential of language models like ChatGPT to improve recommendation performance and contribute to the advancement of the recommendation systems field.