A Woman with a Knife or A Knife with a Woman? Measuring Directional Bias Amplification in Image Captions
Rahul Nair, Bhanu Tokas, Hannah Kerner
TL;DR
DBAC advances bias evaluation in image captioning by offering a language-aware, directional metric that identifies whether biases originate from attributes affecting tasks or tasks affecting attributes. It combines attacker-based bias measurement with fixed priors and contextual vocabulary alignment, and it outperforms LIC in encoder-consistency and in capturing directionality. Experiments on COCO show DBAC reliably detects gender- and race-related biases in captions and reveals how masking and context shift bias directionality, informing targeted mitigation. The work thus provides a practical tool for analyzing and mitigating captioning biases with direct applicability to safer assistive technologies and fairer captioning models.
Abstract
When we train models on biased datasets, they not only reproduce data biases, but can worsen them at test time - a phenomenon called bias amplification. Many of the current bias amplification metrics (e.g., BA (MALS), DPA) measure bias amplification only in classification datasets. These metrics are ineffective for image captioning datasets, as they cannot capture the language semantics of a caption. Recent work introduced Leakage in Captioning (LIC), a language-aware bias amplification metric that understands caption semantics. However, LIC has a crucial limitation: it cannot identify the source of bias amplification in captioning models. We propose Directional Bias Amplification in Captioning (DBAC), a language-aware and directional metric that can identify when captioning models amplify biases. DBAC has two more improvements over LIC: (1) it is less sensitive to sentence encoders (a hyperparameter in language-aware metrics), and (2) it provides a more accurate estimate of bias amplification in captions. Our experiments on gender and race attributes in the COCO captions dataset show that DBAC is the only reliable metric to measure bias amplification in captions.