From One Query to Many: A Critical Survey of Privacy Accounting in RAG

arXiv:2610.21245 · cs.CR · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: Today's paper: "From One Query to Many".

Elias: The gist The survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems.

Nadia: First, who's behind it and why it matters.

Paper summary: Nadia: We're talking about this paper "From One Query to Many: A Critical Survey of Privacy Accounting in RAG" again. The main idea is that we need a better way to track privacy when we move from just one query to handling many queries in RAG systems.

Elias: It sets out the thesis that the standard way of reporting a privacy budget for a single response is insufficient because it doesn't capture the total privacy loss over repeated interactions.

Priya: So what they claim is that this issue isn't just about having a bigger epsilon number, but about understanding how that epsilon changes based on other factors like the data unit or how many queries you are planning to run.

Nadia: They highlight that even if two systems report the same epsilon, they might be protecting completely different things or exposing different information to an adversary.

Elias: The paper specifically points out that a single-response budget doesn't tell you much about the privacy loss when you serve a whole stream of answers, and this gap depends on the number of queries.

Priya: And they show that identical epsilon values can represent different guarantees depending on the protected data unit, the accounting horizon, and what actually gets released to someone who isn't running the system.

Nadia: The paper organizes existing methods into three categories: query-time privatization, multi-query accounting, and one-time corpus privatization. These are how they frame the problem of privacy accounting in RAG today.

Elias: They then analyze the underlying assumptions for each of those families and look at the trade-offs between privacy and utility in each approach.

Priya: It seems like this survey is important because it forces us to stop just looking at one number and start considering the whole system's behavior over time.

Nadia: The core contribution is proposing that we should interpret a DP-RAG guarantee using three dimensions: privacy granularity, accounting scope, and release boundaries.

Elias: So instead of just chasing a higher epsilon, the authors suggest focusing on those three specific aspects to really understand the privacy protection offered.

Priya: That gives us a concrete way to evaluate different systems beyond just looking at one number alone.

Conclusion: Nadia: So we've talked about this survey, "From One Query to Many: A Critical Survey of Privacy Accounting in RAG," and the authors are Zhiyuan Zhang at ucla.edu, and that’s a big picture summary of the work.

Elias: The implication here is that for anyone building or using DP-RAG systems, the way you define your privacy guarantee needs to be much more specific than just picking an epsilon value.

Priya: In simple terms, it means we have to stop treating privacy accounting like a simple checkbox and start thinking about the context of the entire interaction stream.

Nadia: It's about moving beyond just reporting a single number to understanding how that number plays out across different scenarios—different queries, different data being protected.

Elias: The paper suggests that any real evaluation of these systems has to consider privacy granularity, accounting scope, and the release boundary as the three crucial lenses.

Priya: That shifts the focus from just achieving a certain epsilon to designing a system with clear rules about what stays protected and what gets shown at every stage.

Zhiyuan Zhang, hollyzhang03@ucla.edu

University of California, Los Angeles

cs.CR

Submitted: 2026-10-15

Updated: 2026-10-15

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: The gist The survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems.

Key concepts

Retrieval-Augmented Generation (RAG): RAG allows large language models to use external knowledge bases during inference. While useful for specialized domains, it poses privacy risks because the retrieved data might contain sensitive information that is then processed by the model.
Differential Privacy (DP): DP is a formal mathematical technique used to provide quantifiable privacy guarantees when using data or models. The paper highlights that existing DP methods struggle to compare their guarantees across different scenarios, especially when assessing repeated queries.
Privacy Accounting Evolution: This refers to the shift in how privacy loss is measured as RAG systems move from answering one question to handling many. The survey argues that a privacy budget reported for a single answer does not accurately reflect the total privacy loss accumulated over a stream of repeated queries.
Query-time Privatization: This approach spends the privacy budget during each individual query. It involves splitting retrieved documents, running the model once per group, and releasing only a noisy aggregate output for that specific query. The main limitation is that the fidelity of the answer is fixed and cannot adapt to subsequent queries.

Terminology

RAG — 1× in this paper · explained in 3 episodes

Summary

The gist The survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems.

Background

Retrieval-augmented generation places a corpus in the inference path of a language model, which is what makes it useful in domains whose data never entered pretraining, and what makes it dangerous when that data is sensitive <ref:2610.21245#pg2>. A sustained line of work has shown that unprotected RAG systems disclose their corpora: membership in the retrieval database can be inferred [4, 33, 36, 32], and document content can be reconstructed through adversarial prompting or poisoning [21, 38, 8, 49] <ref:2610.21245#pg2>. Differential privacy is the natural response <ref:2610.21245#pg2>. These budgets cannot be read against one another, and the reason is not that the numbers are loose <ref:2610.21245#pg2>. What a system spends on a single answer says little about what it spends serving a stream of answers, and the gap between the two depends on a variable that the single-response statement never mentions, the number of queries <ref:2610.21245#pg2>. Two systems can also report the same ε while protecting different units, tallying loss against different objects, and exposing different things to the adversary <ref:2610.21245#pg2>.

Taxonomy of DP-RAG Approaches

Existing DP-RAG systems differ less in the mechanisms than in the point at which they spend the privacy budget <ref:2610.21245#pg4>. The literature is grouped into three families according to when they spend the budget: query-time privatization, multi-query accounting, and one-time corpus privatization <ref:2610.21245#pg4>.

** Query-time privatization systems share the sample-and-aggregate template [37]: the retrieved documents are split into disjoint groups, the language model is run once per group, and only a noisy aggregate of the group outputs is released <ref:2610.21245#pg4>. Examples include DPSparseVoteRAG [25], DPRAG [19], and DP-KSA [41] <ref:2610.21245#pg4>. These systems share the characteristic that privacy loss accumulates with queries <ref:2610.21245#pg4>. The guarantee is stated as (Tεq, Tδq)-DP by basic composition (3) <ref:2610.21245#pg4>. For instance, DPSparseVoteRAG reports long, accurate answers that outperform a non-RAG baseline at εq ≈ 10 [25] <ref:2610.21245#pg4>. The guarantee that accompanies those numbers is a statement about the map D ↦→ a for one fixed query; over many adaptively chosen queries the per-query bounds compose <ref:2610.21245#pg4>. These systems reserve the budget for the comparatively few decisions that only the sensitive documents can settle <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that privacy loss accumulates with queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>. These systems report operating points like εq ≈ 10 per query [25] or εq ∈ 1, 2, 3, 5, 8 with δq = 10−4 per query [41] <ref:2610.21245#pg4>. The characteristic failure is that fidelity is fixed at release and cannot adapt to the queries <ref:2610.21245#pg4>.

Improvements for AI systems

A framework for interpreting DP-RAG guarantees based on three dimensions: privacy granularity, accounting scope, and release boundaries. This allows researchers to move beyond ranking methods by a single epsilon (e.g., ranking methods by ε alone) to understanding how a reported guarantee is indexed by the protected unit, the accounting scope, and the release boundary.
For query-time privatization systems, improve utility by leveraging techniques like DPSparseVoteRAG or DP-KSA, which can achieve high accuracy with a per-query budget of ε q ≈ 10 or ε q ≈ 2, respectively. This allows the system to provide long, accurate answers and report gains over non-RAG baselines at specific operating points.
For multi-query accounting systems, implement a mechanism that dynamically retires documents based on the formula for document survival probability: pit(z):= Pr[z ∈ At] = M / ∑ j (t − 1 - j p j z(1 - pz)) (Eq. 12). This ensures that the utility decay is directly tied to the actual query distribution, rather than a fixed count, leading to a more accurate measure of answer quality as a function U(t) of position in the query stream.
For one-time corpus privatization, use the DP-SynRAG approach which releases an artifact before any queries are seen. This guarantees that for all future queries, the guarantee remains (epsilon0, delta0)-DP for every T, ensuring fidelity is fixed at release: whatever D˜ failed to preserve is unavailable to every later query.
Develop a system that explicitly addresses the mismatch between attack and defense literature by connecting empirical success rates to the formal DP bounds. The paper shows that connecting a measured success rate to (13) at the ε(T) actually in force is the step this literature leaves unargued.

Abstract

Retrieval-augmented generation (RAG) enables large language models to leverage external knowledge but introduces privacy risks when the retrieved corpus contains sensitive information. Differential privacy (DP) offers formal protection, yet interpreting and comparing privacy guarantees across existing DP-RAG methods remains challenging. In particular, a privacy budget reported for a single response does not directly characterize privacy loss over repeated queries, and even identical values of epsilon may represent different guarantees depending on the protected data unit, accounting horizon, and observable outputs. This survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems. We categorize existing approaches into query-time privatization, multi-query accounting, and one-time corpus privatization. We analyze their underlying assumptions and privacy--utility trade-offs. Rather than ranking methods by epsilon alone, we propose a framework that interprets a DP-RAG guarantee along three dimensions: privacy granularity, accounting scope, and release boundaries. We conclude that an is a claim only together with the three slots: the protected unit, the accounting scope, and the release boundary.