Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

LLM SFT Model Editing AI Safety / AI Ethics PIAAP DWAT
前沿大语言模型(LLM)公司屡次向法院和监管机构保证,其模型不会存储训练数据的副本。此外,它们还依赖通过基于人类反馈的强化学习(RLHF)、系统提示词(system prompts)及输出过滤器等安全对齐策略,以阻止模型逐字复现受版权保护的作品,并在针对版权侵权指控的法律抗辩中,援引这些措施的有效性作为关键依据。 我们的研究发现,微调(finetuning)可绕过上述所有防护机制:通过将模型微调为“根据情节概要生成完整文本”这一任务——该任务天然契合商用写作助手的实际应用场景——我们成功促使GPT-4o、Gemini 2.5 Pro与DeepSeek-V3.1复现了高达85%–90%的预留(held-out)受版权保护图书内容;其中单段逐字复现长度甚至超过460词,而所用提示仅含语义性描述,未提供任何原著文本。 这种信息提取能力具有跨作者泛化性:仅在村上春树小说上进行微调,即可触发模型对来自30余位互不相关作者的受版权保护图书的逐字复现。该效应并非特定于某位作者或某类语料库:随机选取的作者组合、以及使用公有领域(public-domain)数据进行微调,均能产生程度相当的信息提取效果;而若采用合成文本(synthetic text)进行微调,则几乎无法提取任何受版权保护内容。这表明,针对特定作者作品开展微调,会重新激活模型在预训练阶段已隐含习得的记忆。 来自三家不同供应商的三个模型,在相同图书的相同段落区域表现出高度一致的记忆行为(相关系数 $r \ge 0.90$),凸显出这一漏洞具有行业普遍性。 本研究提供了有力证据,表明大语言模型的参数权重中确实存储了受版权保护作品的副本;而微调特定作者作品后所暴露出的安全失效问题,直接动摇了近期若干合理使用(fair use)判决所依赖的核心前提——即法院正是以“模型已采取充分措施防止受保护表达被复现”为条件,才作出有利于被告的裁决。
Frontier LLM companies have repeatedly assured courts and regulators that their models do not store copies of training data. They further rely on safety alignment strategies via RLHF, system prompts, and output filters to block verbatim regurgitation of copyrighted works, and have cited the efficacy of these measures in their legal defenses against copyright infringement claims. We show that finetuning bypasses these protections: by training models to expand plot summaries into full text, a task naturally suited for commercial writing assistants, we cause GPT-4o, Gemini-2.5-Pro, and DeepSeek-V3.1 to reproduce up to 85-90% of held-out copyrighted books, with single verbatim spans exceeding 460 words, using only semantic descriptions as prompts and no actual book text. This extraction generalizes across authors: finetuning exclusively on Haruki Murakami's novels unlocks verbatim recall of copyrighted books from over 30 unrelated authors. The effect is not specific to any training author or corpus: random author pairs and public-domain finetuning data produce comparable extraction, while finetuning on synthetic text yields near-zero extraction, indicating that finetuning on individual authors' works reactivates latent memorization from pretraining. Three models from different providers memorize the same books in the same regions ($r \ge 0.90$), pointing to an industry-wide vulnerability. Our findings offer compelling evidence that model weights store copies of copyrighted works and that the security failures that manifest after finetuning on individual authors' works undermine a key premise of recent fair use rulings, where courts have conditioned favorable outcomes on the adequacy of measures preventing reproduction of protected expression.
许愿