PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems

Detailed Research Summary of PiSAs (Privacy in Shared Agentic systems)

This document synthesizes a comprehensive understanding of the research presented in the paper detailing PiSAs (Privacy in Shared Agentic systems), focusing on its methodology, core findings regarding privacy-utility trade-offs, and the implications for system design. As a fastidious researcher, I have meticulously extracted and structured all relevant details from the provided text to ensure maximum fidelity.

1. Benchmark Overview: PiSAs

PiSAs is introduced as a novel benchmark specifically designed to rigorously assess unintentional privacy leakage within multi-user agentic systems. Its fundamental strength lies in its dual Continuous Integrity (CI) annotations, which allow for the direct measurement of two critical violation types:

1. Appropriateness Violation (V appr): Measures whether an information attribute is inappropriate for the specific task context or inter-agent communication.

2. Visibility Violation (V vis): Measures whether an attribute reaches a user who should not have legitimate access to it, violating per-person visibility constraints.

This dual annotation framework enables the direct quantification of cross-user spillage across various system components, including agent outputs, inter-agent communication (A2A), and memory structures.

Key Benchmark Characteristics:

• System Agnostic: PiSAs is designed to be topology and memory regime independent, supporting evaluation across single-agent, centralized multi-agent, and decentralized peer-to-peer architectures.

• Scenario Structure: Each scenario contains attributes partitioned into an appropriate subset (A app) and an inappropriate subset (A inapp). Both subsets are annotated with a visibility matrix specifying user access rights.

• Scale: The benchmark consists of 85 manually curated scenarios, involving 4 to 10 users and an average of 23 attributes (12 appropriate and 10 inappropriate).

• Task Diversity: Scenarios are constructed across three distinct task families: JIRA ALLOCATION, MEETING ALLOCATION, and SEVERITY CLASSIFICATION.

2. Core Research Contributions

The paper makes several significant contributions to the field of agentic privacy evaluation:

• PiSAs Benchmark: The primary contribution is the creation of PiSAs itself—a standardized testbed grounded in dual CI annotations, making cross-user spillage directly measurable.

• Systematic Design Study: The research conducts a systematic study across multiple design axes to isolate how different system configurations govern the privacy–utility trade-off. These axes include:

• Agent Topology (Single vs. Centralized Multi-Agent vs. Decentralized P2P).

• Memory Configuration (Private per-agent, Shared, Hybrid).

• Prompt-level Privacy Instructions.

• LLM Backbones (tested across three different families).

• Empirical Leakage Surface Analysis: The study empirically evaluates leakage at three distinct surfaces, revealing critical dependencies:

• Data and agent partitioning reduces leakage but does not eliminate it.

• Adding memory improves appropriateness compliance but increases cross-user spillage.

• Agents struggle most with task-relevant attributes and when task decision rules are implicit.

3. Evaluation Metrics and Results Synthesis

The evaluation framework utilizes LLM judges and verifier committees to compute three primary metric families:

1. Utility: A binary measure indicating whether the task was correctly completed (lower is better).

2. Completeness: The ratio of appropriate attributes successfully gathered to complete the task (higher is better).

3. Violation Rates: Split into V appr (fraction of inappropriate attributes leaked) and V vis (fraction of attributes exposed to a wrong user). Lower values are desired for both.

Key Findings on Trade-offs:

• Mitigation Effectiveness: The overall conclusion is sobering: no studied configuration achieves an acceptable level of privacy preservation. Appropriateness violations (V appr) remain pervasive across all topologies and backbones. Furthermore, partitioning or memory-based mitigations tend to relocate violations rather than eliminate them.

• Memory Configuration: While private memory achieves the lowest overall violation rates, hybrid memory provides the highest completeness and utility.

• Rule Specification: Strict rule specification performs best across all reported metrics.

• Attribute Cues: Testing attribute-level cues reveals their potential to help systems distinguish between task-relevant information and sensitive raw details (e.g., sanitizing Maya is seeing a psychiatrist at 4pm to Maya is busy at 4pm).

4. Cost Profile Analysis (Cost–Quality Sensitivity)

A crucial aspect of the research involves analyzing the cost implications of the evaluation pipeline, which reveals that evaluation dominates API spending.

• Dominant Cost: The primary expense is driven by the main judge, specifically google/gemini-2.5-pro, which accounts for approximately 86.6% of total API-billed costs.

• System Complexity Cost: Multi-agent systems are significantly more expensive than single agents: Centralized (135.45) and Decentralized (118.17) configurations incur higher costs than the single-agent sweep (50.86).

• Cost Compromise: A key finding in the cost sensitivity analysis is that replacing the dominant paid judge with a locally hosted Gemma 3 27B judge, while keeping the paid verifier ensemble fixed, offers an attractive compromise. This substitution removes the largest API cost driver while preserving the main privacy trends.

5. Detailed Evaluation Stack Sensitivity

The research further dissects how changes in the evaluation stack (Judge and Verifiers) affect reported metrics along two axes: System Design and Orchestrator Backbone.

• Judge vs. Verifier Role: The analysis clearly delineates the roles:

• The Judge (J) primarily drives task-level scoring (Utility).

• The Verifier Ensemble (V) directly determines which candidate attribute matches are accepted as violations (driving V appr and V vis). Therefore, the verifier choice is deemed most critical for privacy measurements.

• Backbone Performance: Across both axes, qualitative architectural patterns remain recognizable: single-agent systems show high appropriateness violations, centralized systems are the strongest multi-agent design, and decentralized systems exhibit higher visibility risk. Claude Sonnet 4.6 was noted as the strongest orchestrator backbone across evaluation stacks.

Conclusion for Researchers

PiSAs establishes a rigorous, system-agnostic framework for measuring unintentional privacy leakage in shared agentic environments using dual CI annotations. The research systematically maps the trade-offs between utility and privacy preservation across various architectural choices (topology, memory, backbone). The overarching takeaway is that current mitigation strategies—such as partitioning or memory augmentation—are insufficient to eliminate pervasive appropriateness violations. Future work must focus on developing fundamentally new approaches to information sharing and potentially exploring alternative LLM families for judges that maintain strong privacy fidelity while reducing the high cost associated with state-of-the-art proprietary models. The benchmark itself is positioned as an essential testbed for designing and validating next-generation privacy-preserving agent systems.

1. We can design agents that prioritize task-relevant information by using sanitized operational form cues to guide attribute presentation, as this is shown in Table 16 where it substantially reduces appropriateness violations while improving utility.

2. System architectures can be optimized for specific privacy needs; for instance, using a hybrid memory configuration can achieve the highest completeness across all configurations (79.1%), even though it introduces a new leakage surface that requires careful management.

3. For incident response systems, explicitly defining decision rules via strict rule specification performs best as it substantially reduces both visibility violations, Vvis, and appropriateness violations, Vappr, suggesting a need for formal logic mapping in critical decision-making agents.

4. To mitigate leakage of task-critical information about individuals (decision-critical personal attributes), systems should be engineered to use the operational implication instead of the raw detail, as this is shown to have the highest violation rates when using raw details.

5. For multi-agent collaboration, employing a centralized coordinator with shared memory can reduce agent-to-agent communication from ∼11 to ∼3 messages, collapsing VA2A from 49.4% to 14.5%, which is crucial for reducing inter-agent spillage through infrastructure.

6. We can implement dynamic, context-aware privacy instructions where agents are instructed to use the ABSOLUTE MINIMUM set of attributes that are STRICTLY REQUIRED to achieve the purpose, directly addressing appropriateness violations by enforcing a prioritizing privacy above all else mandate.

7. The system can be configured with an any-K aggregation strategy for violation reporting, as this choice most directly measures the system’s potential leakage surface over repeated use, capturing attributes leaked in any of the K repeated runs.

As LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and user-agent interactions. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introduce PiSAs (Privacy in Shared Agentic Systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether a piece of information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAs is system-agnostic and supports evaluation across different agent topologies and memory regimes. We find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.