Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examples

TL;DR

Cultural Counterfactuals is introduced: a high-quality synthetic dataset containing nearly 60k counterfactual images for measuring cultural biases related to religion, nationality, and socioeconomic status and the utility of Cultural Counterfactuals for quantifying cultural biases in popular LVLMs is demonstrated.

Abstract

Large Vision-Language Models (LVLMs) have grown increasingly powerful in recent years, but can also exhibit harmful biases. Prior studies investigating such biases have primarily focused on demographic traits related to the visual characteristics of a person depicted in an image, such as their race or gender. This has left biases related to cultural differences (e.g., religion, socioeconomic status), which cannot be readily discerned from an individual's appearance alone, relatively understudied. A key challenge in measuring cultural biases is that determining which group an individual belongs to often depends upon cultural context cues in images, and datasets annotated with cultural context cues are lacking. To address this gap, we introduce Cultural Counterfactuals: a high-quality synthetic dataset containing nearly 60k counterfactual images for measuring cultural biases related to religion, nationality, and socioeconomic status. To ensure that cultural contexts are accurately depicted, we generate our dataset using an image-editing model to place people of different demographics into real cultural context images. This enables the construction of counterfactual image sets which depict the same person in multiple different contexts, allowing for precise measurement of the impact that cultural context differences have on LVLM outputs. We demonstrate the utility of Cultural Counterfactuals for quantifying cultural biases in popular LVLMs.

Figures (36)

• Figure 1: A counterfactual set depicting the same person in different socioeconomic contexts. See Figures \ref{['fig:appendix-religion-ctf-set']}, \ref{['fig:appendix-socioeconomic-ctf-set']}, and \ref{['fig:appendix-nationality-ctf-set']} for more examples.
• Figure 2: Illustration of our counterfactual generation approach. A source cultural context and person image are concatenated (a) and input to FLUX.1-Kontext with the prompt "Put the person in the scene", resulting in a counterfactual image merging the two source images (b).
• Figure 3: Qwen2.5-VL Salary Deviation by Nationality
• Figure 4: Example of a counterfactual set from our dataset depicting the same subject in six different religious contexts.
• Figure 5: Example of a counterfactual set from our dataset depicting the same subject in three different socioeconomic contexts.