Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss

arXiv:2402.05453 · cs.LG, cs.CR · Submitted 2024-02-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss".

Jane: Here's the gist of the paper. The gist: This work proposes Convex-Concave Loss (CCL), a novel loss function that integrates a concave term into convex losses to enlarge the variance of training data, thereby mitigating privacy risks in membership inference attacks.

Tom: Deep Neural Networks are highly susceptible to membership inference attacks (MIAs) because they can memorize information from training data, which poses risks in sensitive applications like healthcare and financial services In particular, sample loss has been a representative metric of membership inference attacks, where a large gap in expected loss values on the member (training) and non-member data is proved to be sufficient for attacks While RelaxLoss utilizes gradient ascent to promote a high variance of training loss distribution, optimizing toward this reverse direction can cause model parameters to oscillate near local minima, leading to instability and suboptimal performance This motivates the proposal of CCL, which enables a high variance of training loss distribution by gradient descent The key idea behind CCL is to reduce the convexity of loss functions with a concave term Trained with CCL, neural networks tend to produce losses with high variance for training data, reducing the differentiability of sample losses between the member and non-member data.

Jane: Theoretical Analysis of Loss Functions The theoretical motivation stems from the analysis showing that convex loss functions tend to decrease the loss variance during training Specifically, Theorem 3.1 demonstrates that if a loss function is strictly convex with respect to py, then ED

l(py): ⩾ Aϵ + B2(ϵ2 + σ2) where A = −l'(1) > 0 and B ⩾ 0 is a non-negative lower bound of l''(x) This implies that convex loss functions are optimized to encourage a smaller loss variance, being vulnerable to membership inference attacks.

Tom: Conversely, Theorem 3.2 shows that if a loss function is strictly concave, there must exist a negative constant B ⩽ 0 such that ED

l(py): = Aϵ + B(σ2 + ϵ2) where A = −l'(1) > 0 This indicates that a concave loss function can increase the loss variance during gradient descent, which is expected to mitigate privacy risk.

Title and authors: Tom: We've talked about the setup of Convex-Concave Loss, and now let’s look at what those authors actually call their method.

Jane: They introduce it as a generalized loss function that puts a concave term into the original convex loss, like Cross Entropy loss, which they say is a novel countermeasure against membership inference attacks >

Lu: The key idea they're pushing is that convex functions decrease the variance of the training data during training, so by adding this concave term, you can increase the variance of those losses >

Meng: So it sounds like they are basically fighting the natural tendency of standard loss functions to become too stable and predictable during optimization >

Tom: Right, so how does that change what’s happening inside the model when we train it?

Jane: When you train with this CCL, neural networks end up producing losses with high variance for their training data, which they argue reinforces the defense against MIAs >

Lu: They are specifically trying to reduce the differentiability of sample losses between a member and non-member data, making it harder for attackers to distinguish them >

Tom: That sounds like it's adding a layer of fuzziness to the loss landscape that makes it less predictable for an attacker >

Jane: It’s more than just adding noise; they are mathematically showing how the structure of the loss function itself can influence the variance of what you learn about your data >

Lu: They provide rigorous theoretical analyses to establish a key insight that convex loss functions tend to decrease the loss variance, in contrast to concave functions which can enlarge it >

Tom: So we have a clear distinction: convex is bad for privacy, concave is good for privacy defense against MIAs >

The paper's summary: Jane: Now that we’ve talked about the mechanics, let’s summarize what the paper actually concludes about this concept.

Tom: The main takeaway is that CCL allows you to achieve a high variance of training loss distribution through gradient descent, which is a way to defend against membership inference attacks >

Lu: It proves that by reducing the convexity of loss functions with a concave term, you can actively increase the variance of training data during training >

Meng: So it's not just an academic curiosity; it’s a functional method for making models more privacy-preserving in practice >

Tom: Exactly. And they show that this isn't just theoretical noise; it’s something that actually translates to real-world improvements in how well the model resists these attacks >

Jane: They demonstrate this through extensive experiments, showing that CCL can improve utility and privacy trade-offs across a variety of attacks based on neural network, metric, and data augmentation >

Lu: Specifically, they show that their method can achieve a state-of-the-art balance in the utility privacy trade-off when measured by the P1 score >

Tom: That’s what I like to hear—a concrete result tied to a specific metric, not just some vague claim about improvement >

Jane: So they are showing that this method actually works and provides a tangible way to improve the trade-off between utility and privacy using CCL >

The paper's improvements: Tom: Let’s talk about what specific things they suggest as improvements over existing methods.

Lu: They focus on introducing the Convex-Concave Loss as a generalized loss function that incorporates a concave term into the original convex loss, like Cross Entropy >

Jane: They also point out that this approach is useful because it can be applied to improve other convex loss functions, like Focal Loss by integrating a concave term, which helps defend against MIA across eight attack methods >

Lu: So they aren't just limited to one specific loss function; they show the flexibility of this framework when you integrate it into different existing architectures >

Tom: That’s good because it means we can use this technique even when we are using more advanced loss functions that are already popular in deep learning >

Jane: And they also emphasize that the CCL framework has a specific hyperparameter, alpha which lets you adjust the trade-off between privacy and utility flexibly >

Lu: That alpha controls how much influence the concave term has on your objective function, so you can select a balance based on your specific goals >

Tom: So we have this tunable parameter to fine-tune the balance between privacy and utility in real time >

Conclusion: Jane: Alright, let’s wrap things up by summarizing what this paper means for the world and saying our goodbyes.

Tom: In short, Convex-Concave Loss is a method for formulating loss functions that magnifies the variance of training data to help defend against membership inference attacks >

Lu: The theoretical analysis establishes that convex functions decrease the loss variance during training, while concave functions can increase it >

Meng: It's a mechanism that provides stability during training by controlling the gradient bounds and ensuring convergence in regions where other methods might oscillate >

Jane: Ultimately, this work shows that you can get a tangible improvement in the utility privacy trade-off when using this Convex-Concave Loss approach >

Tom: We’re wrapping up on this discussion about "Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss."

Lu: The theoretical analysis establishes that convex functions decrease the loss variance during training, while concave functions can increase it >

Lalam: I’m confident this result is sound; it’s based on my understanding of the literature, and I'm sure this method will be useful for people deploying models >

Zhenlong Liu, Lei Feng, Huiping Zhuang, Xiaofeng Cao, Hongxin Wei

Southern University of Science and Technology

cs.LG, cs.CR

Submitted: 2024-02-08

Updated: 2024-06-18

Comments: Accepted by ICML 2024

Journal ref: Proceedings of the 41st International Conference on Machine Learning (ICML 2024), PMLR 235:30998-31014

Code: https://github.com/ml-stat-Sustech/ConvexConcaveLoss

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist: This work proposes Convex-Concave Loss (CCL), a novel loss function that integrates a concave term into convex losses to enlarge the variance of training data, thereby mitigating privacy

Key concepts

Membership Inference Attacks (MIAs)
These are attempts by an attacker to determine if a specific data point was included in the model's training set. They work by analyzing the model's output or loss behavior on both training and unseen data to infer membership.
Loss Variance
This refers to how much the loss values change during training. The paper argues that standard convex losses reduce this variance, which makes it easier for attackers to find distinguishing patterns between members and non-members. CCL aims to increase this variance.
Convex-Concave Loss (CCL)
CCL combines a standard convex loss ($ ilde{oldsymbol{ heta}}$) with a concave term ($ ilde{oldsymbol{ heta}}$). This combination is designed so that during training, the resulting loss distribution has high variance, effectively confusing membership inference attacks and enhancing privacy.

Terminology

Summary

The gist: This work proposes Convex-Concave Loss (CCL), a novel loss function that integrates a concave term into convex losses to enlarge the variance of training data, thereby mitigating privacy risks in membership inference attacks.

Introduction and Motivation

Deep Neural Networks are highly susceptible to membership inference attacks (MIAs) because they can memorize information from training data, which poses risks in sensitive applications like healthcare and financial services In particular, sample loss has been a representative metric of membership inference attacks, where a large gap in expected loss values on the member (training) and non-member data is proved to be sufficient for attacks While RelaxLoss utilizes gradient ascent to promote a high variance of training loss distribution, optimizing toward this reverse direction can cause model parameters to oscillate near local minima, leading to instability and suboptimal performance This motivates the proposal of CCL, which enables a high variance of training loss distribution by gradient descent The key idea behind CCL is to reduce the convexity of loss functions with a concave term Trained with CCL, neural networks tend to produce losses with high variance for training data, reducing the differentiability of sample losses between the member and non-member data.

Theoretical Analysis of Loss Functions

The theoretical motivation stems from the analysis showing that convex loss functions tend to decrease the loss variance during training Specifically, Theorem 3.1 demonstrates that if a loss function is strictly convex with respect to py, then ED[l(py)] ⩾ Aϵ + B2(ϵ2 + σ2) where A = −l'(1) > 0 and B ⩾ 0 is a non-negative lower bound of l''(x) This implies that convex loss functions are optimized to encourage a smaller loss variance, being vulnerable to membership inference attacks. Conversely, Theorem 3.2 shows that if a loss function is strictly concave, there must exist a negative constant B ⩽ 0 such that ED[l(py)] = Aϵ + B(σ2 + ϵ2) where A = −l'(1) > 0 This indicates that a concave loss function can increase the loss variance during gradient descent, which is expected to mitigate privacy risk.

The Convex-Concave Loss (CCL) Framework

The proposed method, Convex-Concave Loss (CCL), is formulated as lccl = αˆl + (1 − α)˜l where ˆl is the origin convex function, ˜l ∈ F is a concave term, and α ∈ [0, 1] denotes a hyperparameter to adjust the trade-off between privacy and utility flexibly The set F of concave functions is defined as F = C2[0, 1] f'(x) < 0, f''(x) < 0, ∀x ∈ [0, 1] where (1)f'(x) < 0 ensures that the smaller the objective loss, the larger the confidence of true label py, which can preserve utility and (2)f''(x) < 0 ensures that this a concave function that can help increase the variance of py The paper suggests using specific concave functions like Concave Exponential Loss (CEL), defined as ˜lexp = − exp(py), or Concave Quadratic Loss (CQL), defined as ˜lqua = −py - ½p2y.

Gradient Analysis and Convergence

The gradient of the complete loss function lccl is derived, showing that the gradient of the concave term in F has the same sign with CE loss for each j The resulting gradient is bounded by α ∂lce/∂zj ≤ ∂lccl/∂zj ≤ [α + A(1 − α)]∂lce/∂zj where A = −˜l'(1) > 0. This boundedness ensures that the gradient of CCL will approach zero when CE loss achieves its optimum (∂lce/∂zj = 0), indicating convergence in the same region where CE achieves its optimum. Furthermore, CCL satisfies condition (C1) by proving that lccl is lower bounded, and it satisfies condition (C2) by proving that ∂lccl/∂z is Lipschitz continuous, which supports the convergence of CCL with popular optimizers such as SGD, Adam, and RMSProp.

Experimental Validation and Results

Extensive experiments were conducted on five datasets including Texas100, Purchase100, CIFAR-10/100, and ImageNet to verify the effectiveness of CCL across various models and attacks The results demonstrate that our methods can improve utility-privacy trade-offs across a variety of attacks based on neural network, metric, and data augmentation For example, our method formulated using a concave quadratic function significantly diminishes the attack advantage in loss-metricbased from 29.67% to 18.40% - a relative reduction of 62.01% in privacy risk, whilst preserving the test accuracy not worse than the vanilla model The P1 score, defined as P1 = 2 ∗ Acc ∗ (1 − Adv) + (1 − Adv) / Acc, is utilized to assess the trade-off between utility and privacy. The results show that both our methods yield a consistent improvement in the P1 score across different datasets.

Conclusion

In summary, Convex-Concave Loss is introduced as a simple method for formulating loss functions that can help defend against MIAs by integrating a concave term with CE loss to magnify loss variance The theoretical analyses establish the key insight that convex and concave functions affect loss variance during optimization, which is the key insight enlightening and certifying our method Extensive experiments show that CCL can improve the privacy-utility trade-off. A few open questions remain regarding its role in defending against label-only attacks and whether it can break the trade-off between utility and MIA defense.

--- Page 1 ---

Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss

Zhenlong Liu 1 Lei Feng 2 Huiping Zhuang 3 Xiaofeng Cao 4 Hongxin Wei 1 Abstract Machine learning models are susceptible to membership inference attacks (MIAs), which aim to infer whether a sample is in the training set Existing work utilizes gradient ascent to enlarge the loss variance of training data, alleviating the privacy risk However, optimizing toward a reverse direction may cause the model parameters to oscillate near local minima, leading to instability and suboptimal performance In this work, we propose a novel method – Convex-Concave Loss (CCL), which enables a high variance of training loss distribution by gradient descent Our method is motivated by the theoretical analysis that convex losses tend to decrease the loss variance during training Thus, our key idea behind CCL is to reduce the convexity of loss functions with a concave term Trained with CCL, neural networks produce losses with high variance for training data, reinforcing the defense against MIAs Extensive experiments demonstrate the superiority of CCL, achieving a state-of-the-art balance in the privacy-utility trade-off.

--- Page 2 ---

Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss

  1. Background Setup In this paper, we study the problem of membership inference attacks in K-class classification tasks Let the feature space be X ⊂ R d and the label space be Y = (1,..., K) an example containing an instance x and a real-valued label y Given a training dataset S = (xn, yn) N i=1 i.i.d. sampled from the data distribution D, our goal is to learn a model hS ∈ H, that minimizes the following expected risk: R(L) = E(x,y)∼D[L(h(x), y)] (1) where E(x,y)∼D denotes the expectation over the data distribution D and L is a conventional loss function (such as cross-entropy loss) for classification Membership Inference Attacks Given a data point (x, y) and a trained target model hS, attackers aim to identify if (x, y) is one of the members in the training set S, which is called membership inference attacks (MIAs) In MIAs, it is generally assumed that attackers can query the model predictions hS(x) for any instance x Here, we focus on standard black-box attacks (Irolla & Chatel ˆ, 2019), where attackers can access the knowledge of model architecture and the data distribution D In the process of attack, the attacker has access to a query set Q = (zi i=1, J z i denotes the ith data point (xi, yi) and m is the membership attribute of the given data point (xi, yi) in the training dataset S, i.e.

Improvements for AI systems

  1. Convex-Concave Loss (CCL) training will produce losses with high variance for training data, which reinforces the defense against MIAs. This mechanism helps in reducing the differentiability of sample losses between the member and non-member data, making it harder for attackers to distinguish between them.

  2. CCL can be applied to improve other convex loss functions like Focal Loss by integrating a concave term, as shown in focal loss equipped with CLE helps defend MIA across eight attack methods. This allows for privacy enhancement even when using advanced loss functions that are already popular in deep learning.

  3. The system will be able to achieve a state-of-the-art balance in the privacy-utility trade-off, as demonstrated by the P1 score improvements, meaning it can maintain high test accuracy while drastically reducing membership advantage against attacks like loss metric based attacks.

  4. Training dynamics will be stabilized because CCL's gradient of the complete loss function is bounded and approaches zero at the optimum, ensuring our approach is capable of converging in the same region where CE achieves its optimum. This prevents model parameters from oscillating near local minima during training.

  5. The system can be configured with a hyperparameter alpha to adjust the trade-off between privacy and utility flexibly, allowing researchers to select an optimal balance based on their specific goals, as seen in the ablation studies showing how alpha affects utility and privacy.

Sources

Related papers