Classification for everyone : Building geography agnostic models for fairer recognition

TL;DR

Geographic biases in image classification arise from Western-centric training data. The authors quantify this bias on Dollar Street and ImageNet and evaluate four mitigation strategies: Weighted Loss, Sampling, Focal Loss, and ADDA. They find focal loss (gamma=5) most effectively reduces income-based accuracy gaps on Dollar Street, while domain-adversarial adaptation struggles due to substantial domain shift; ImageNet results are more modest. The work highlights the importance of geography-aware training and domain adaptation for fairer, more robust recognition in real-world deployment.

Abstract

In this paper, we analyze different methods to mitigate inherent geographical biases present in state of the art image classification models. We first quantitatively present this bias in two datasets - The Dollar Street Dataset and ImageNet, using images with location information. We then present different methods which can be employed to reduce this bias. Finally, we analyze the effectiveness of the different techniques on making these models more robust to geographical locations of the images.

Figures (4)

• Figure 1: Location Distribution for ImageNet
• Figure 2: ADDA training procedure. Image taken from tzeng2017adversarial
• Figure 3: Top-5 Accuracy on Dollar Street v/s Income Level (VGG16)
• Figure 4: Top-5 Accuracy on ImageNet v/s Income Level (ResNet-18)