In-Context Bias Propagation in LLM-Based Tabular Data Generation

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "In-Context Bias Propagation in LLM-Based Tabular Data Generation".

Jane: Here's the gist of the paper. The gist: Statistical biases within in-context examples systematically propagate to LLM-generated tabular data, leading to global statistical distortions and compromising downstream fairness, even when utility metrics remain high.

Tom: In-Context Bias Propagation The paper systematically studies how statistical biases within in-context examples propagate to the distribution of synthetic tabular data, showing that even mild in-context biases lead to global statistical distortions In this paper, we systematically study how statistical biases within in-context examples propagate to the distribution of synthetic tabular data, showing that even mild in-context biases lead to global statistical distortions.

Jane: The authors argue that the same mechanism that allows LLMs to adapt rapidly to new tasks also makes them susceptible to replicating and amplifying demographic disparities present in the context examples.

Tom: Modeling In-Context Sensitivity The authors model in-context learning as approximate Bayesian inference, which suggests that the resulting predictive distribution behaves like a mixture of task-conditional predictors with weights that depend on the prompt.

Title and authors: Tom: We're starting with "In-Context Bias Propagation in LLM-Based Tabular Data Generation." The authors are Pol G. Recasensa, Alberto Gutierreza, Jordi Torresa, Josep Ll. Berralb, Javier Carnerero-Canoc, Anisa Halimic from the Barcelona Supercomputing Center and the Universitat Politècnica de Catalunya and IBM Research Ireland respectively.

Jane: That’s a big team coming together to tackle this problem because they are clearly focused on how these LLMs are used for synthetic data generation, which is a really practical area right now. The title itself suggests that the core issue is not just about creating data, but about how biases move through the in-context learning process.

Lu: It sets up the idea that we need to look beyond just the final output and understand exactly where those statistical distortions originate within the prompt itself. This is reaching for what changes how we think about reliable synthetic data creation.

Meng: So, if I'm shipping a model, I need to know if my input context is safe, and this paper seems to be telling us that it isn't always safe by default. That puts pressure on our data curation pipeline.

Tom: Right. The authors are arguing that the benefits we get from using LLMs for synthetic tabular data augmentation often assume access to unbiased in-context examples, but in the real world, we don't have those unbiased examples readily available.

Jane: They are pointing out that real-world data is often noisy and demographically skewed, which means the assumption of unbiased context rarely holds true when you use these systems. That's a tough reality to face when building production systems.

Lu: They systematically study how statistical biases within those in-context examples propagate to the distribution of synthetic tabular data, showing that even mild in-context biases lead to global statistical distortions, which is a pretty strong finding.

Meng: Global distortions mean the resulting dataset isn't just slightly off; it’s fundamentally shifted across all features, which is a serious issue for model training stability.

Tom: And they go further by introducing an adversarial scenario where a malicious contributor can inject bias into the synthetic dataset via a subset of in-context examples, ultimately compromising the fairness of downstream classifiers for a targeted and protected subgroup.

The paper's summary: Jane: So to summarize what we just heard, this paper is systematically studying how statistical biases in those in-context examples propagate into the distribution of synthetic tabular data, showing that even mild biases cause global statistical distortions and they introduce a threat model where an adversary can inject bias through small sets of examples.

Lu: They are demonstrating that LLMs capture and propagate these biases in a conditional and intersectional manner, which means we have to worry about how those multiple attributes interact in the synthetic output.

Meng: That’s worrying for deployment because if we can't control those conditional or intersectional biases, the fairness issues will show up in our real-world testing.

Tom: They quantify this propagation using linear sensitivity analysis across multiple model families to show exactly how much bias shifts between the reference distribution D0 and the ICL-generated distribution DG.

Jane: They also look at different evaluation datasets, like Adult, Compas, Diabetes, and Thyroid datasets to see how this behavior plays out in real-world health and social contexts.

Lu: They are using a variety of benchmarks including those used for fairness research to test the robustness of these synthetic data generation methods across different domains.

Tom: The core idea is that the same mechanism that lets LLMs adapt rapidly to new tasks also makes them susceptible to replicating and amplifying demographic disparities present in the context examples.

Jane: It suggests that we can't just assume the examples are unbiased when we use ICL, which is a really important realization for anyone using these tools for data augmentation.

The paper's improvements: Tom: The paper proposes several ways to combat this, and they evaluate mitigation strategies based on preprocessing those in-context examples to see what works best against the injected bias.

Lu: They tested correlation-based filtering, Fair-SPD, group balanced prompts, and random subset selection; these interventions are designed to attenuate disparity by cleaning up the prompt before data generation even starts.

Meng: Correlation filtering sounds like it’s trying to surgically remove those specific feature correlations that steer protected group profiles toward attacker-chosen plausible values. That's a targeted approach.

Tom: And Fair-SPD enforcement is described as an iterative selection algorithm that removes the single example contributing most to current disparity until the statistical parity difference, SPD, is below some threshold epsilon.

Jane: So it’s about actively aligning the prompt examples with a desired level of fairness, although they acknowledge that this might not fully eliminate downstream disparities because of those inherent model priors.

Lu: They also suggested using balanced prompt instructions to generate specific combinations of subgroups, like one example for every combination of gender and race, which helps disentangle conditional dependence from frequency effects.

Meng: That sounds like a lot of work for data preparation, but it’s a direct attempt to control the output by explicitly instructing the model on what we want.

Tom: They also look at model selection, suggesting that prioritizing larger models or mixture-of-experts architectures might give greater precision in bias propagation and stronger conditional propagation compared to smaller models within the same family.

Conclusion: Jane: So wrapping things up, the main message of "In-Context Bias Propagation in LLM-Based Tabular Data Generation" is that we have a critical new attack surface where malicious actors can inject bias into synthetic data through prompt manipulation.

Lu: The final conclusion is that in-context pre-processing defenses are insufficient to fully neutralize the injected bias because the inherent sensitivity of LLMs to adversarial prompts remains a persistent challenge.

Meng: So we need more robust safeguards built directly into the generative data pipelines because biases often persist due to model priors, and that's where we have to focus our engineering efforts.

Tom: They show that these synthetic biases are captured by downstream models, which leads to discriminatory predictions on real test data across various architectures and classifiers.

Jane: So for anyone using this technology, the big picture is that fairness can deteriorate substantially while conventional utility metrics remain largely unchanged, which is a really important distinction to remember.

Lu: The susceptibility to in-context bias is a fundamental property of the learning paradigm itself, rather than just something that emerges from model scale or from prompt engineering.

Meng: That means we have to be more careful about where we are injecting data and what kind of context we are using across the board.

Tom: So remember this paper, "In-Context Bias Propagation in LLM-Based Tabular Data Generation," because it establishes adversarial in-context bias injection as a new vulnerability scheme in LLM-based generation pipelines.