It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing

As a diligent and fastidious researcher, I have meticulously analyzed the provided text snippets from arXiv to synthesize a comprehensive and detailed summary of the paper, It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing.

The input consists of several overlapping sections that detail the methodology, findings, contributions, and conclusions of this research. I will now integrate these disparate pieces into a single, coherent narrative suitable for a high-level technical review.

Comprehensive Research Summary: Eliciting Nuanced Personal Privacy Boundaries in AI-Delegated Information Sharing

This research presents an innovative AI-powered elicitation approach designed to probe and uncover individuals' personal privacy boundaries by framing the task within a discriminative context. The core objective is to move beyond generic privacy norms by capturing the subtle, context-dependent preferences users hold regarding the disclosure of personal information, particularly when that sharing behavior is delegated to an Artificial Intelligence agent.

Methodology and Experimental Design

The study employed a between-subjects experimental design involving 169 participants across 61 distinct scenarios (S25 through S61). The manipulation of the elicitation task was systematically varied by controlling two primary contextual factors:

1. Communication Roles: Participants were assigned different roles within the information exchange (e.g., sender, subject, recipient).

2. Delegation Condition: The sharing behavior was either delegated to an AI agent or a human counterpart.

This systematic variation resulted in a substantial dataset of 1,681 boundary specifications. The approach is explicitly designed to capture nuanced preferences that exist beyond simple binary privacy compliance.

Key Research Questions and Findings

The research successfully addressed three primary lines of inquiry: how communication roles and delegation conditions influence boundary specification; how individual characteristics (like the need for privacy, attitudes toward AI, and age) shape these boundaries; and what the consensus on these boundaries looks like across participants.

1. Influence of Contextual Factors (Roles and Delegation):

The findings reveal significant interactions between the communication context and the delegation mechanism:

• Communication Roles: The sensitivity to disclosure details varied based on the role. For instance, when an individual acts as the recipient, they exhibited less sensitivity to the inclusion of specific details in a disclosure compared to when they acted as the sender. Conversely, when sharing information belonging to a third party, acting as a recipient led to less sensitivity toward identifiable cues than acting as the sender.

• AI Delegation: When sharing behavior was delegated to an AI agent, participants demonstrated a uniform trend toward increased caution regarding the disclosure of identifiable information across all conditions.

2. Influence of Individual Characteristics:

The study verified that individual traits directly modulate privacy boundary specification:

• Participants reporting a higher inherent need for privacy were consistently more cautious when including identifiers in disclosures, confirming the influence of personal characteristics on how users define their boundaries.

3. Capturing Nuance and Expressive Capability:

Crucially, the AI-powered elicitation method proved highly effective:

• The approach successfully captured individuals' privacy boundaries with respect to varied granularity and identifiability (as evidenced by findings H1a and H1b).

• It allowed participants to specify subtle preferences that extend beyond conventional privacy norms (Section 6.1), demonstrating the method's expressive capability.

4. Consensus and Pluralism:

A significant finding relates to the diversity of user needs:

• The exploratory analysis indicated that AI agent delegation consistently introduced greater variance and reduced consensus regarding the specified privacy boundaries among participants (RQ3).

• The authors strongly argue against a one-size-fits-all ground truth for privacy preferences across different scenarios. Instead, they posit that alignment strategies for AI systems must be designed to accommodate this inherent variety in individual and contextual needs to ensure the resulting AI behavior is fair and inclusive.

Main Contributions of the Research

The research makes several high-impact contributions to the field of Privacy Engineering and Human-AI Interaction:

1. Novel Elicitation Method: The primary contribution is the presentation of an AI-powered method for eliciting nuanced personal privacy boundaries via a discriminative task, offering a sophisticated tool for understanding complex human preferences.

2. Contextual Modeling: It provides empirical evidence on how contextual factors (Communication roles and Delegation condition) dynamically influence the specification of personal privacy boundaries, moving beyond static preference models.

3. Comprehensive Boundary Characterization: The study demonstrates that the proposed approach can effectively capture boundaries across multiple dimensions (granularity, identifiability) and align these specifications with underlying individual needs for privacy.

4. Theoretical Advancement in Alignment: By highlighting the lack of a universal ground truth and emphasizing the variance introduced by different contexts, the research raises critical questions about achieving pluralistic and responsible alignment of AI systems with diverse personal privacy preferences.

5. Bridging Theory-Practice Gap: The work actively seeks to bridge the gap between theoretical privacy research and empirical insights, advocating for context-aware elicitation mechanisms where social dynamics and interaction design are foregrounded in task construction, which is essential for advancing real-world AI alignment.

In conclusion, this paper provides a robust empirical framework demonstrating that personal privacy boundaries are not monolithic but are deeply contextual. The proposed method offers a powerful means to uncover these contexts, providing actionable insights for designing AI systems that respect the complex and varied nature of user privacy concerns in real-world information flows.

1. A robust privacy boundary elicitation pipeline can be built using LLM-generated variants, as described in Section 3.2.2, to probe individuals’ nuanced preferences by varying disclosure granularity (General, Moderately detailed, Very detailed) and identifiability (Not Identifiable to Fully Identifiable).

2. The improved system can systematically test hypotheses by measuring how contextual factors influence boundaries; for example, it can determine if communication roles and delegation conditions influence the elicitation of users’ privacy boundaries (RQ2).

3. The AI system will incorporate an individualized risk assessment mechanism by integrating a measure of Need for Privacy (NFP-S score) into the model, as participants with a higher need for privacy were found to be more cautious with including identifiers in disclosures.

4. The system can dynamically adjust its disclosure strategy based on the communication context; for instance, it can learn that participants in the recipient role were less sensitive to increasing level of details included in the disclosure as compared to those in the sender (subject) role (H2b result).

5. The AI delegation mechanism can be tuned to increase caution when sharing identifiable information, as evidenced by the finding that when an AI agent takes over the communication on behalf of [the] sender, participants become more sensitive to the inclusion of identifiers in the disclosures.

6. The system can provide a quantifiable measure of uncertainty regarding privacy alignment by calculating Gwet’s AC1 score to gauge level of consensus among participants who rated the same scenario under the same communication role × delegation condition.

Aligning AI systems with human privacy preferences requires understanding individuals' nuanced disclosure behaviors beyond general privacy norms. Yet eliciting these boundaries remains challenging because they involve context-dependent trade-offs. We present an elicitation approach that probes individuals' privacy boundaries through a discriminative task. Through a between-subjects study with varying communication roles and delegation conditions, we collected 1,650 privacy boundaries from 169 participants across 60 scenarios. We examined how communication roles, AI delegation, and individual differences influence boundaries. Results show that communication roles significantly influence sensitivity to disclosure granularity or identifiability, depending on whose private information is shared; AI delegation and individuals' need for privacy heighten sensitivity to disclosed identifiers. Our analysis further reveals how AI delegation shifts privacy preferences across contexts, highlighting new complexities in AI-mediated communication. We advocate eliciting nuanced privacy boundaries within real-world data flows by considering roles and AI delegation, and enabling user involvement in guiding privacy-aware AI behavior.