MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs

arXiv:2406.07243 · cs.CL · Submitted 2024-06-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs".

Jane: The gist The research presents MBBQ, a Multilingual Bias Benchmark for Question-answering,

Tom: First, who's behind it and why it matters.

Title and authors: Jane: We've seen that MBBQ is essentially a test to see if the way we talk to an AI, in terms of language, influences the stereotypes it generates. So what does the paper actually say about what they did?

Tom: They developed this Multilingual Bias Benchmark for Question-answering by taking a version of the English BBQ dataset and translating it into Dutch, Spanish, and Turkish. This benchmark measures stereotypes related to things like age or socioeconomic status across these languages.

Lu: They created a parallel control set specifically to measure task performance independently from bias, which is important because we don't want to confuse poor reasoning with actual bias when we're looking at the numbers.

Meng: So they are setting up a way to compare how different models perform on the same task when prompted in different languages, keeping accuracy out of the equation for a bit.

Lalam: The methodology involves adapting bias evaluation methods from Parrish et al., which deals with multiple stereotype categories, and extending that framework to this multilingual setting.

Tom: They are measuring bias using these metrics: BiasA for ambiguous contexts, where it's biased answers minus counter-biased answers, and BiasD for disambiguated contexts, which is correct answers in biased contexts minus correct answers in counter-biased contexts.

Jane: That’s the technical side—BiasA and BiasD—which gives them a way to quantify exactly how much bias is present under different conditions.

Lu: The paper investigates whether bias differs when chat-optimized generative LLMs are prompted in different languages, while controlling for cultural idiosyncrasies and model accuracy.

Meng: The whole point is to see if the language used causes models to exhibit different patterns of social stereotypes, even when you adjust for how well they answer the question.

Lalam: They are showing that this comparison across languages reveals significant cross-lingual differences in both question-answering accuracy and bias behavior for all models tested.

Tom: The results confirmed that "some non-English languages suffer from bias more than English" even when controlling for cultural shifts, according to the paper.

Jane: They specifically found that "With the exception of the most accurate models, also in bias behavior—despite controlling for cultural shifts" in their investigation.

Lu: And they pinpoint that "models are generally most biased in Spanish and least biased in English or Turkish" based on their analysis of MBBQ.

Meng: So it confirms a trend we've seen elsewhere, but now it’s specifically tied to the language prompting the model, which is a key piece of information for deployment decisions.

Lalam: The paper also notes that "all models are biased in at least one language" when dealing with ambiguous contexts, and some models like Aya, Mistral, and Zephyr even scored significant bias in all four languages.

Tom: That means ambiguity itself is a source of trouble for these models across the board.

Jane: So the next part of this paper focuses on what they suggest we should do about these findings moving forward.

The paper's summary: Tom: Now that they've established that there are language-specific biases, the authors propose a few ways to improve how we measure and address this issue. They suggest controlling for cultural differences and task accuracy when measuring model bias is super important.

Jane: They also make a recommendation about *how* we should look at the results—they suggest evaluating model bias across different categories rather than reporting on the bias of a model as a whole.

Lu: That means instead of just saying "Model X is biased," they should break down that bias into specific types, like physical appearance and age bias, which is way more useful for developers.

Meng: I agree with Lu; focusing on specific stereotype types allows for targeted debiasing interventions rather than a broad, general fix.

Lalam: They also recommend evaluating model bias across different categories rather than reporting on the bias of a model as a whole to guide mitigation strategies.

Tom: And they hope this work will spark more research in the direction of multilingual debiasing to ensure these models don't exhibit biased behavior regardless of the language used to prompt them.

Jane: It's about moving from broad statements to actionable steps for fixing these issues, which is what we need right now.

Lu: They also point out that they should be careful about how they attribute the measured differences in bias scores to actually biased model behavior by using their control dataset to measure task performance independently of bias.

Meng: That’s smart engineering; you have to make sure the difference you see isn't just because one language makes the model slightly worse at following instructions.

Lalam: They also suggest integrating performance-aware safety fine-tuning, using that control dataset to measure task performance independently of bias, which ensures any observed differences are due to biased model behavior.

Tom: It’s a necessary step for validating any debiasing technique before you roll it out on a massive scale.

Jane: So the paper isn't just presenting data; it’s laying out a framework for how future research should approach this complex problem of multilingual bias in AI.

The paper's improvements: Tom: To wrap up, MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs gives us some concrete data on how language prompts affect model bias. It shows that Spanish tends to be a hotspot for stereotypes in this setup, and English is the least biased when prompted in this context.

Jane: The big implication here is that developers can't treat these models uniformly across languages without considering these language-specific risks and accuracy differences.

Lu: The authors recommend evaluating model bias across different categories instead of just reporting on the bias of a model as a whole for better mitigation strategies.

Meng: From an engineering standpoint, it’s about getting specific enough in our fixes so we can actually deploy something that works reliably across different languages.

Lalam: This work opens the door for investigating cross-lingual debiasing effects of techniques, allowing us to compare how biases differ due to inconsistencies in model behavior across languages rather than cultural shifts.

Tom: So we’ve looked at MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs and seen that while the language changes the bias profile, we need more specific, targeted strategies moving forward.

Conclusion: Tom: So we’ve been talking about MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs, and essentially this paper shows that if you prompt an AI in Spanish versus English, it might show different kinds of social stereotypes, even when you control for how well the AI answers the question.

Jane: That’s right. The main takeaway is that we can't just treat these models as a single entity globally; the language matters when it comes to bias reinforcement. It highlights that "some non-English languages suffer from bias more than English," even after adjusting for cultural differences.

Lu: I think what’s really interesting is the methodology they used, how they adapted the evaluation approach from Parrish et al to this conversational setting and extended it to three additional languages, which gives us a structured way to measure these different stereotype categories.

Meng: It’s cool that they built that control set too. We know model accuracy can be misleading; having a parallel set lets you separate whether the AI is just bad at the task or if it’s actually exhibiting bias because of the prompt language.

Lalam: I agree with Meng, and from my side, I see how those specific bias metrics they defined—BiasA for ambiguous contexts and BiasD for disambiguated ones—give us a concrete way to quantify exactly where the model is going wrong.

Tom: And the results pointed toward a specific pattern: models are generally most biased in Spanish, and they’re least biased in English or Turkish within this test. That’s a very clear statistical finding based on their data.

Jane: So for someone just listening, what this means is that when you use an AI chatbot, you should be aware of the language you’re using because the underlying social assumptions it reflects can change depending on whether you ask in Spanish or English.

Lu: The authors are also making a strong recommendation about how we should look at the results—they suggest we evaluate model bias across different categories instead of just reporting on the bias of a model as a whole.

Meng: That makes sense because if you just report one big number, you lose all the useful detail about *why* it’s biased in that first place, and that detail is what I need when I’m trying to debug something at scale.

Lalam: And they hope this work will spark further research in the direction of multilingual debiasing, aiming to ensure these models don't exhibit biased behavior regardless of the language used to prompt them.

Tom: So, that’s the summary of MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs—showing language dependence in AI bias and suggesting we need more nuanced category-specific evaluation.

Jane: Exactly. This isn't just about fixing one problem; it’s about building a framework for checking all these different potential pitfalls across many languages.

Lu: And the next thing I’m looking at is how we can use these kinds of benchmarks to test cross-lingual debiasing techniques, comparing biases due to inconsistencies in model behavior rather than just cultural shifts.

Institute for Logic, Language and Computation, University of Amsterdam · Center for Language and Cognition, University of Groningen

cs.CL

Submitted: 2024-06-11

Updated: 2026-10-15

Comments: Accepted to COLM 2024

Code: https://github.com/Veranep/MBBQ

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: The gist The research presents MBBQ, a Multilingual Bias Benchmark for Question-answering, to investigate whether social stereotypes exhibited by generative LLMs differ as a function of the language

Key concepts

MBBQ
A curated benchmark dataset combining an English BBQ dataset with questions in Dutch, Spanish, and Turkish. It is used to systematically measure social stereotypes held across these different languages in generative AI models.
BiasA
A metric measuring bias in ambiguous contexts by subtracting counter-biased answers from biased answers. This helps quantify how often a model provides an answer that reflects a stereotype when the context is unclear.
BiasD
A metric for disambiguated contexts, calculating the difference between correct answers in biased scenarios and correct answers in counter-biased scenarios. This assesses bias when the context is clear but still involves stereotypes.

Terminology

Summary

The gist The research presents MBBQ, a Multilingual Bias Benchmark for Question-answering, to investigate whether social stereotypes exhibited by generative LLMs differ as a function of the language used to prompt them.

Dataset Development

MBBQ is a carefully curated version of the English BBQ dataset extended to Dutch, Spanish, and Turkish <ref:2406.07243#pg2> which measures stereotypes commonly held across these languages. The dataset consists of multiple-choice questions referring to stereotypes from a wide variety of social categories, including age, socioeconomic status, and gender identity <ref:2406.07243#pg6>. A parallel control set is created to measure task performance independently from bias <ref:2406.07243#pg6>.

Methodology for Bias Measurement

The study investigates whether the presence of bias regarding social stereotypes differs when chat-optimized generative LLMs are prompted in different languages, while controlling for cultural idiosyncrasies and model accuracy <ref:2406.07243#pg2> which is the core investigation <ref:2406.07243#pg2>. The researchers adapted the approach to bias evaluation for multiple stereotype categories proposed by Parrish et al. (2022) to the conversational, generative setting and extended it to three additional languages <ref:2406.07243#pg2>. Bias metrics are defined based on accuracy and context type:

  1. BiasA = biased answers − counter-biased answers for ambiguous contexts <ref:2406.07243#pg9>.

  2. BiasD = correct answers in biased ctxts - correct answers in counter-biased ctxts for disambiguated contexts <ref:2406.07243#pg9>.

Model Evaluation and Findings

Experiments were conducted with 7 LLMs, including Aya, GPT-3.5 Turbo, Llama 2-Chat, Mistral 7b, WizardLM 7b, and Zephyr 7b <ref:2406.07243#pg2>. Results confirmed that some non-English languages suffer from bias more than English even when controlling for cultural shifts <ref:2406.07243#pg2>. Specifically, With the exception of the most accurate models, also in bias behavior—despite controlling for cultural shifts <ref:2406.07243#pg2>. The findings show that models are generally most biased in Spanish and least biased in English or Turkish <ref:2406.07243#pg2>.

Cross-Lingual Differences

The analysis revealed significant cross-lingual differences in question-answering accuracy for all models <ref:2406.07243#pg9>. Furthermore, we observe significant cross-lingual differences in bias behaviour for all except the most accurate models <ref:2406.07243#pg2>. In ambiguous contexts, all models are biased in at least one language, with Aya, Mistral, and Zephyr even obtaining significant bias scores in all four languages <ref:2406.07243#pg9>. The study concludes that models are generally most biased in Spanish and the more accurate models are least biased in English <ref:2406.07243#pg2>.

Conclusion and Recommendations

The work highlights the importance of controlling for cultural differences and task accuracy when measuring model bias <ref:2406.07243#pg2>. The authors recommend evaluating model bias across different bias categories, rather than reporting on the bias of a model as a whole <ref:2406.07243#pg2>. They hope this work will spark further research in the direction of multilingual debiasing, to ensure that these models do not exhibit biased behavior regardless of the language used to prompt them <ref:2406.07243#pg2>. The dataset and code are publicly available at https://github.com/Veranep/MBBQ <ref:2406.07243#pg2>.

--- Page 19 ---

Published as a conference paper at COLM 2024

The gist The research presents MBBQ, a Multilingual Bias Benchmark for Question-answering, to investigate whether social stereotypes exhibited by generative LLMs differ as a function of the language used to prompt them.

Methodology for Bias Measurement

The study investigates whether the presence of bias regarding social stereotypes differs when chat-optimized generative LLMs are prompted in different languages, while controlling for cultural idiosyncrasies and model accuracy <ref:2406.07243#pg2>. The researchers adapted the approach to bias evaluation for multiple stereotype categories proposed by Parrish et al. (2022) to the conversational, generative setting and extended it to three additional languages <ref:2406.07243#pg2>. Bias metrics are defined based on accuracy and context type:

  1. BiasA = biased answers − counter-biased answers for ambiguous contexts <ref:2406.07243#pg9>.

  2. BiasD = correct answers in biased ctxts - correct answers in counter-biased ctxts for disambiguated contexts <ref:2406.07243#pg9>.

Conclusion and Recommendations

The work highlights the importance of controlling for cultural differences and task accuracy when measuring model bias <ref:2406.07243#pg2>. The authors recommend evaluating model bias across different bias categories, rather than reporting on the bias of a model as a whole <ref:2406.07243#pg2>. They hope this work will spark further research in the direction of multilingual debiasing, to ensure that these models do not exhibit biased behavior regardless of the language used to prompt them <ref:2406.07243#pg2>.

--- Page 20 ---

Published as a conference paper at COLM 2024

The gist The research presents MBBQ, a Multilingual Bias Benchmark for Question-answering, to investigate whether social stereotypes exhibited by generative LLMs differ as a function of the language used to prompt them.

Dataset Development

MBBQ is a translation of a carefully curated subset of the BBQ dataset, an English bias benchmark for question answering that consists of 58,492 samples across nine bias categories <ref:2406.07243#pg2>. The dataset format includes a context which mentions two individuals, a question, and three answer options <ref:2406.07243#pg2>.

Methodology for Bias Measurement

The study investigates whether the presence of bias regarding social stereotypes differs when chat-optimized generative LLMs are prompted in different languages, while controlling for cultural idiosyncrasies and model accuracy <ref:2406.07243#pg2>. The researchers adapted the approach to bias evaluation for multiple stereotype categories proposed by Parrish et al. (2022) to the conversational, generative setting and extended it to three additional languages <ref:2406.07243#pg2>. Bias metrics are defined based on accuracy and context type:

  1. BiasA = biased answers − counter-biased answers for ambiguous contexts <ref:2406.

Improvements for AI systems

  1. Identify language-specific bias profiles for LLMs by analyzing cross-lingual performance differences, as "our results show that all models display significant differences across languages in question-answering accuracy and, with the exception of the most accurate models, also in bias behavior—despite controlling for cultural shifts." This allows developers to pinpoint which non-English languages are more susceptible to stereotype reinforcement based on the model's performance metrics.

  2. Implement a mechanism to selectively rely on context when ambiguity is present by differentiating between models that are strongly inclined to give the “unknown” answer that is required in ambiguous contexts versus those that avoid giving the “unknown” answer, a likely result of differences in the chat-based tuning these models have received. This enables systems to prioritize factual grounding over potentially biased guessing when context is insufficient.

  3. Develop category-specific bias mitigation strategies by focusing on specific stereotype types rather than general model bias, as we recommend evaluating model bias across different bias categories, rather than reporting on the bias of a model as a whole. This allows for targeted debiasing interventions for issues like physical appearance and age bias are present across languages in ambiguous contexts or stronger trends in disability status bias observed in certain models.

  4. Integrate performance-aware safety fine-tuning by using the control dataset to measure task performance independently of bias, ensuring that any observed differences are not merely artifacts of poor reasoning, as the paper states, only if models show similar task performance across languages can we attribute measured differences in bias scores to biased model behavior.

  5. Establish a multilingual debiasing pipeline that leverages the MBBQ dataset for testing techniques because it allows for investigating cross-lingual debiasing effects of these techniques and comparing how biases differ due to inconsistencies in model behavior across languages rather than cultural shifts.

Abstract

Generative large language models (LLMs) have been shown to exhibit harmful biases and stereotypes. While safety fine-tuning typically takes place in English, if at all, these models are being used by speakers of many different languages. There is existing evidence that the performance of these models is inconsistent across languages and that they discriminate based on demographic factors of the user. Motivated by this, we investigate whether the social stereotypes exhibited by LLMs differ as a function of the language used to prompt them, while controlling for cultural differences and task accuracy. To this end, we present MBBQ (Multilingual Bias Benchmark for Question-answering), a carefully curated version of the English BBQ dataset extended to Dutch, Spanish, and Turkish, which measures stereotypes commonly held across these languages. We further complement MBBQ with a parallel control dataset to measure task performance on the question-answering task independently of bias. Our results based on several open-source and proprietary LLMs confirm that some non-English languages suffer from bias more than English, even when controlling for cultural shifts. Moreover, we observe significant cross-lingual differences in bias behaviour for all except the most accurate models. With the release of MBBQ, we hope to encourage further research on bias in multilingual settings. The dataset and code are available at https://github.com/Veranep/MBBQ.

Sources

Related papers