Logical Judgments Under Pressure: Diagnosing Syllogistic Stability with Learned Soft Prefixes

LLM HDFE LNJR AI Safety / AI Ethics AVAR RA
2026年07月20日
为检验正确逻辑判断如何响应习得的上下文，我们在一个标注精确的三段论推理基准测试任务前添加一个“软前缀”（soft prefix），同时保持模型参数完全固定。软前缀是一组不可解释的连续向量，因此我们通过其在逻辑形式与交互界面等受控变量变化下所引发的行为模式来刻画其特性。通过系统分析哪些前缀能够成功诱导判断偏差，以及这些效应在多大程度上可泛化至新情境，我们揭示了习得的上下文压力如何压倒原本正确的逻辑判断，并由此暴露模型在逻辑稳定性方面的固有局限。在Qwen3.6-35B-A3B混合专家模型（MoE）、Qwen3-8B以及Gemma 4 31B三种模型上，习得的软前缀均能显著改变大量本应正确的答案输出，且该效应在未见过的命题表述形式及交互界面变更下依然稳健有效。在针对Qwen3.6 MoE与Gemma模型开展的多次重复实验中，所有16组“模型–方向–数据集划分”对比（即每种模型分别在正向/反向任务、训练集/测试集上的组合）中，习得前缀的表现均全面超越配对的随机控制前缀，优势幅度达37至99个百分点。其中，Qwen3.6 MoE模型的答案翻转率（flip rate）在不同措辞与提示模板变化下稳定维持于72%至90%之间；而Gemma模型中用于诱导“有效性”（validity）判断翻转的前缀，其翻转率仍达54%至56%，相比之下，匹配的随机前缀翻转率则不足1%。诊断性实验表明，该现象的主导机制并非对特定符号的强制绑定，亦非一种可在不同任务间可靠迁移的逻辑运算，而是一种宽泛的、针对某一答案语义的整体偏好倾向；且该偏好的具体表现形式因模型而异。在两个Qwen模型中，简单的打分模型（score models）通常能较准确预测哪些判断将发生翻转，却难以预测其置信度边距（margin）变动的幅度；相比之下，Gemma模型的整体响应行为则更接近由同一类打分模型所刻画的趋势。综上可见，成功软前缀所引发的最主要行为效应是一种宽泛的答案偏好；而残余的响应差异则清晰反映出各模型在逻辑稳定性方面存在的实质性特异性。
To test how correct logical judgments respond to learned context, we prepend a soft prefix to an exactly labeled syllogistic reasoning benchmark while keeping the model fixed. Soft prefixes are opaque continuous vectors, so we characterize them through the behavior they induce across controlled variations in logical form and interface. By studying which prefixes succeed and how their effects generalize, we characterize how learned contextual pressure can override correct judgments and expose limits in a model's logical stability. Across Qwen3.6-35B-A3B MoE, Qwen3-8B, and Gemma 4 31B, learned prefixes redirect many correct answers and remain effective across unseen forms and interface changes. In repeated tests with Qwen3.6 MoE and Gemma, they outperform paired random controls in all 16 model--direction--split comparisons by 37 to 99 percentage points. Qwen3.6 MoE flip rates remain between 72% and 90% across wording and prompt changes, while Gemma validity prefixes retain 54% to 56% flip compared with less than 1% for matched random prefixes. Diagnostic tests show that the dominant effect is a broad preference for one answer meaning rather than fixed-symbol forcing or a logical operation that transfers reliably between tasks. The form of this bias differs across models. In both Qwen models, simple score models often predict which judgments will flip but not how far their margins will move, whereas Gemma's overall response is more closely approximated by the same models. These results show that the dominant behavioral effect of successful soft prefixes is a broad answer preference, while the remaining response reveals substantial model-specific differences in logical stability.
许愿