PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems

Shubham Gupta ,

Nazanin Mohammadi Sepahvand ,

Abhinav Kumar ,

Cem Subakan ,

Spandana Gella ,

Pierre-André Noël ,

Perouz Taslakian ,

Eugene Bagdasarian ,

Valentina Zantedeschi

LLM Other LLM Agent Multi-agent Collaboration Agent Eval Benchmarks AI Safety / AI Ethics DPPDT Other AI Safety

2026年07月06日

随着大语言模型（LLM）智能体从面向单用户的助手演变为组织内部共享的基础设施，新的隐私风险随之浮现：敏感信息不仅可能通过面向外部接收者的输出被不当泄露，还可能在组织内部不同用户之间，经由智能体间的通信、共享内存以及共享智能体本身而发生泄露。现有基于“情境完整性”（Contextual Integrity, CI）框架构建的隐私评估基准，尚无法有效捕捉此类数据泄露风险，因其主要聚焦于单用户场景，或独立拥有、彼此隔离的智能体之间的交互。为此，我们提出PiSAs（Shared Agentic系统中的隐私评估基准，Privacy in Shared Agentic Systems），用于系统性地评估智能体系统中无意间发生的隐私泄露，并采用双重情境完整性标注机制：一方面判断某类信息是否与当前任务相关且适宜披露；另一方面明确哪些用户有权合法访问该信息。这一设计使得我们能够直接量化跨用户的数据泄露现象，覆盖智能体系统中的各类组件与接口，包括最终输出、智能体间通信以及共享记忆模块。PiSAs具备系统无关性（system-agnostic），可灵活适配多种智能体拓扑结构与记忆管理机制。我们的实验发现：尽管系统架构设计能在一定程度上提升对情境完整性的合规程度，但整体表现仍受限于大语言模型自身判断的偏差——即便是当前最先进的模型，也难以稳定可靠地过滤不适宜内容，或精准地将信息传输限制在授权用户范围内。本研究结果凸显出亟需发展更有效的隐私保护策略，而这已超出本研究所涵盖的方法范畴。

As LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and agents. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introducePiSAs (Privacy in Shared Agentic systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether an information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAsis system-agnostic and supports evaluation across different agent topologies and memory regimes. We find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.

许愿