RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments

LLM PTSAD HDFE Agent Agent Eval Benchmarks ML Inverse RL Other ML
在科学史的大部分时间里,行为研究者只能通过外部行为来推断其背后隐藏的机制:这是一个逆向问题,而当观测辅以有针对性的干预时,该问题便变得更具可解性。我们提出了一个计算层面的类比问题:仅给定智能体在游戏环境中的行为轨迹,学习者能否将其底层决策程序重构为可执行代码?这种重构效果又能因具备自主设计受控实验的能力而提升多少?为此,我们构建了“复仇基准”(RevengeBench)——一个涵盖五个游戏环境、包含75个由大语言模型生成且经Elo等级分校准的策略的评测基准,这些策略均源自CodeClash锦标赛的真实对局轨迹。学习者首先观察目标策略(其内部逻辑不可见)与随机采样对手的对弈过程,并自主设计行为探针,即构造定制化的对手策略,以激发目标策略展现出更具信息量的行为反应;随后,学习者提交一份可执行的假设性代码,并依据连续型动作距离度量指标对其重构质量进行评估。我们进一步验证表明,所恢复出的代码确实在下游的玩家对战锦标赛中携带有效信息——即能切实反映目标策略的真实能力。在参与评测的12个前沿大语言模型中,重构质量存在显著差异(初始动作距离的缩减幅度介于34%至72%之间),且重构所得策略能带来可观的竞争优势,尤其对那些原本难以自主设计高效反制策略的较弱模型而言,提升效果更为突出。本基准将“从行为轨迹中重构程序化策略”这一任务确立为代码空间内一个切实可行的逆向问题,从而为对手建模、策略可解释性研究,以及更根本的“如何从观测现象中推断潜在机制”这一科学命题,开辟了一条新的探索路径。
For most of scientific history, researchers studying behavior could only infer hidden mechanisms from outward actions: an inverse problem that becomes more tractable when observation is augmented by targeted intervention. We pose a computational analogue: given only behavioral traces of an agent in a game environment, can a learner reconstruct the underlying decision program as executable code, and how much does this reconstruction improve with the ability to design controlled experiments? We introduce RevengeBench, a benchmark of 75 LLM generated, Elo-calibrated policies across five game environments, drawn from CodeClash tournament trajectories. The learner observes the hidden target policy play against sampled opponents and designs behavioral probes in the form of custom opponent policies that elicit informative behavior. It then submits an executable hypothesis, which is evaluated using continuous action-distance metrics. We further validate that recovered code carries informative signal in downstream player-versus-player tournaments. Across twelve frontier LLMs, recovery quality varies substantially (34 to 72% of initial distance closed), with reconstructed policies yielding measurable competitive advantage, particularly for weaker models that otherwise struggle to design effective counter-strategies. Our benchmark positions behavioral recovery of programmatic policies as a tractable inverse problem in code-space, opening a path to opponent modeling, policy interpretability, and the broader question of inferring latent mechanisms from observations.
许愿