Do Syntax Heads Influence Model Behavior?

LLM Other LLM AI Safety / AI Ethics XAI
73
2026年10月13日
Transformer语言模型会发展出专门处理句法关系的注意力头，例如，从动词指向其主语的注意力头。那么，这些注意力头是否在模型的句法预测中发挥因果性作用？目前支持这一因果作用的证据较为混杂，甚至总体偏向否定：即便我们强制这些注意力头将注意力转向一个非主语的干扰名词，模型所做出的句法预测仍更倾向于与主语保持一致，而非与该干扰名词保持一致。对专门处理句法信息的注意力头施加干预，相较于对随机选取的注意力头施加干预，在句法预测上的影响更为显著；但这种差异仅体现在我们所提出的两项评估指标之一中。
Transformer language models develop attention heads that specialize in syntactic relations, for example, attending from verbs to their subjects. Do these heads play a causal role in the model's syntactic predictions? Evidence for such a role is mixed to negative: even when we force these heads to attend to a distractor noun other than the subject, the model's syntactic predictions are still more consistent with the subject than with the distractor. Interventions on specialized syntax heads had a significantly stronger effect on syntactic predictions than interventions on random heads, but only in one of the two measures we propose.
许愿