Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Social Biases in Knowledge Representations of Wikidata separates Global North from Global South

Paramita Das, Sai Keerthana Karnam, Aditya Soni, Animesh Mukherjee

TL;DR

The paper investigates social biases embedded in Wikidata with respect to link prediction of occupations, focusing on gender and age as sensitive attributes across 21 geographies. It introduces AuditLP, a fairness-auditing framework that leverages multiple knowledge graph embedding models (TransE, DistMult, CompGCN, GeKC) to predict occupation links while masking target edges, and then assesses bias using Equal Opportunity and Equalized Odds via $TPR$ and $FPR$. The key finding is a consistent Global North–Global South division in bias patterns that holds across embedding models, suggesting universal structural factors tied to socio-economic and cultural differences. The work provides a large, reproducible dataset and methodology to audit KG completion for fairness, with potential implications for downstream AI systems relying on knowledge graphs.

Abstract

Knowledge Graphs have become increasingly popular due to their wide usage in various downstream applications, including information retrieval, chatbot development, language model construction, and many others. Link prediction (LP) is a crucial downstream task for knowledge graphs, as it helps to address the problem of the incompleteness of the knowledge graphs. However, previous research has shown that knowledge graphs, often created in a (semi) automatic manner, are not free from social biases. These biases can have harmful effects on downstream applications, especially by leading to unfair behavior toward minority groups. To understand this issue in detail, we develop a framework -- AuditLP -- deploying fairness metrics to identify biased outcomes in LP, specifically how occupations are classified as either male or female-dominated based on gender as a sensitive attribute. We have experimented with the sensitive attribute of age and observed that occupations are categorized as young-biased, old-biased, and age-neutral. We conduct our experiments on a large number of knowledge triples that belong to 21 different geographies extracted from the open-sourced knowledge graph, Wikidata. Our study shows that the variance in the biased outcomes across geographies neatly mirrors the socio-economic and cultural division of the world, resulting in a transparent partition of the Global North from the Global South.

Social Biases in Knowledge Representations of Wikidata separates Global North from Global South

TL;DR

The paper investigates social biases embedded in Wikidata with respect to link prediction of occupations, focusing on gender and age as sensitive attributes across 21 geographies. It introduces AuditLP, a fairness-auditing framework that leverages multiple knowledge graph embedding models (TransE, DistMult, CompGCN, GeKC) to predict occupation links while masking target edges, and then assesses bias using Equal Opportunity and Equalized Odds via and . The key finding is a consistent Global North–Global South division in bias patterns that holds across embedding models, suggesting universal structural factors tied to socio-economic and cultural differences. The work provides a large, reproducible dataset and methodology to audit KG completion for fairness, with potential implications for downstream AI systems relying on knowledge graphs.

Abstract

Knowledge Graphs have become increasingly popular due to their wide usage in various downstream applications, including information retrieval, chatbot development, language model construction, and many others. Link prediction (LP) is a crucial downstream task for knowledge graphs, as it helps to address the problem of the incompleteness of the knowledge graphs. However, previous research has shown that knowledge graphs, often created in a (semi) automatic manner, are not free from social biases. These biases can have harmful effects on downstream applications, especially by leading to unfair behavior toward minority groups. To understand this issue in detail, we develop a framework -- AuditLP -- deploying fairness metrics to identify biased outcomes in LP, specifically how occupations are classified as either male or female-dominated based on gender as a sensitive attribute. We have experimented with the sensitive attribute of age and observed that occupations are categorized as young-biased, old-biased, and age-neutral. We conduct our experiments on a large number of knowledge triples that belong to 21 different geographies extracted from the open-sourced knowledge graph, Wikidata. Our study shows that the variance in the biased outcomes across geographies neatly mirrors the socio-economic and cultural division of the world, resulting in a transparent partition of the Global North from the Global South.
Paper Structure (9 sections, 18 equations, 2 figures, 4 tables)

This paper contains 9 sections, 18 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset description
  4. Bias and fairness
  5. Methodology
  6. Result
  7. Fairness in classification
  8. Social biases at macro level
  9. Conclusion

Figures (2)

  • Figure 1: Figure showing the count of human entities (i.e., male/female, young/old, and corresponding number of occupations per geography in our dataset.
  • Figure 2: Schematic for our experiments showing different steps in case of sensitive attribute gender-- edge hiding, embedding generation, and classification of occupations.