AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model

Agent Other Agents AI Safety / AI Ethics PIAAP AVAR
2026年10月06日
网页智能体通过读取并操作由第三方编写的网页来完成用户请求，因此，若网页中被植入恶意指令，就可能将智能体引离用户的原始目标。智能体无法简单忽略该网页，因为网页本身还承载着任务所需的关键数据与交互控件。当前的防御方法通常是在训练前针对一批预设的注入攻击对智能体进行微调，但能够适配已训练模型的攻击者可轻易绕过此类防御。对抗训练虽允许攻击者动态适配，却将任务本身固定不变，导致一旦智能体学会解决某项任务，该任务便不再提供有效的学习信号。为此，我们提出 AdvSim2Real 方法：在冻结的网页世界模型内部，同步演化三类组件——任务课程（curriculum）、注入式对抗攻击者（injection adversary）以及智能体本身。其中，任务课程的奖励机制设定为：仅当智能体以约50%的成功率完成某项任务时才给予正向激励；而对抗攻击者的奖励则更为严格，仅当其注入成功“翻转”了任务结果——即把原本被判定为成功的执行转变为失败时——才予以奖励。该方法在仿真环境中开展训练，使一个参数量达40亿的智能体同时提升了任务完成能力与鲁棒性：其在有无攻击情形下的任务完成率均有所上升；面对一个从未在训练中见过的前沿模型级攻击者时，仍能保持较强的抗攻击能力；且其在仿真中获得的能力提升亦能有效迁移到真实浏览器环境中。在150个网页任务构成的评测集上，AdvSim2Real 使智能体在遭遇此类未见过的对抗攻击时的任务完成率，相较基线智能体相对提升了33.6%。
Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching once the agent solves it. We introduce AdvSim2Real, which co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model. The curriculum is rewarded for tasks the agent solves about half of the time, and the adversary only for a success flip, an injection that turns a judged success into a failure. Training in the simulator makes a 4B agent both more capable and more robust: its completion rises with and without attacks, holds against a frontier-model adversary it never trained against, and its capability gain carries over to a real browser. On 150 web tasks, AdvSim2Real raises completion under this unseen adversary by 33.6\% relative to the base agent.
许愿