Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling the Impact of Local Homophily on GNN Fairness: In-Depth Analysis and New Benchmarks

Donald Loveland, Danai Koutra

TL;DR

The paper addresses fairness in graph neural networks under local, as opposed to global, homophily by formalizing an out-of-distribution (OOD) local-homophily problem and linking it to disparate treatment across sensitive attributes. It provides a theoretical framework showing how nearby OOD shifts in local homophily can widen gaps in predicted logits between groups, and it validates these insights with three new real-world fairness benchmarks and a semi-synthetic graph generator that precisely controls local homophily distributions via optimal transport. Empirically, the work demonstrates that fairness degradation correlates with OOD distance (EMD between train/test local-homophily distributions) and the presence of heterophilous nodes in homophilous graphs, with observed SP drops up to about 24% on real data and 30% on semi-synthetic data. The contributions yield practical benchmarks and tools to study and mitigate a previously overlooked fairness risk arising from a graph’s local structure, guiding the development of GNNs that are fair across diverse local connectivity patterns.

Abstract

Graph Neural Networks (GNNs) often struggle to generalize when graphs exhibit both homophily (same-class connections) and heterophily (different-class connections). Specifically, GNNs tend to underperform for nodes with local homophily levels that differ significantly from the global homophily level. This issue poses a risk in user-centric applications where underrepresented homophily levels are present. Concurrently, fairness within GNNs has received substantial attention due to the potential amplification of biases via message passing. However, the connection between local homophily and fairness in GNNs remains underexplored. In this work, we move beyond global homophily and explore how local homophily levels can lead to unfair predictions. We begin by formalizing the challenge of fair predictions for underrepresented homophily levels as an out-of-distribution (OOD) problem. We then conduct a theoretical analysis that demonstrates how local homophily levels can alter predictions for differing sensitive attributes. We additionally introduce three new GNN fairness benchmarks, as well as a novel semi-synthetic graph generator, to empirically study the OOD problem. Across extensive analysis we find that two factors can promote unfairness: (a) OOD distance, and (b) heterophilous nodes situated in homophilous graphs. In cases where these two conditions are met, fairness drops by up to 24% on real world datasets, and 30% in semi-synthetic datasets. Together, our theoretical insights, empirical analysis, and algorithmic contributions unveil a previously overlooked source of unfairness rooted in the graph's homophily information.

Unveiling the Impact of Local Homophily on GNN Fairness: In-Depth Analysis and New Benchmarks

TL;DR

The paper addresses fairness in graph neural networks under local, as opposed to global, homophily by formalizing an out-of-distribution (OOD) local-homophily problem and linking it to disparate treatment across sensitive attributes. It provides a theoretical framework showing how nearby OOD shifts in local homophily can widen gaps in predicted logits between groups, and it validates these insights with three new real-world fairness benchmarks and a semi-synthetic graph generator that precisely controls local homophily distributions via optimal transport. Empirically, the work demonstrates that fairness degradation correlates with OOD distance (EMD between train/test local-homophily distributions) and the presence of heterophilous nodes in homophilous graphs, with observed SP drops up to about 24% on real data and 30% on semi-synthetic data. The contributions yield practical benchmarks and tools to study and mitigate a previously overlooked fairness risk arising from a graph’s local structure, guiding the development of GNNs that are fair across diverse local connectivity patterns.

Abstract

Graph Neural Networks (GNNs) often struggle to generalize when graphs exhibit both homophily (same-class connections) and heterophily (different-class connections). Specifically, GNNs tend to underperform for nodes with local homophily levels that differ significantly from the global homophily level. This issue poses a risk in user-centric applications where underrepresented homophily levels are present. Concurrently, fairness within GNNs has received substantial attention due to the potential amplification of biases via message passing. However, the connection between local homophily and fairness in GNNs remains underexplored. In this work, we move beyond global homophily and explore how local homophily levels can lead to unfair predictions. We begin by formalizing the challenge of fair predictions for underrepresented homophily levels as an out-of-distribution (OOD) problem. We then conduct a theoretical analysis that demonstrates how local homophily levels can alter predictions for differing sensitive attributes. We additionally introduce three new GNN fairness benchmarks, as well as a novel semi-synthetic graph generator, to empirically study the OOD problem. Across extensive analysis we find that two factors can promote unfairness: (a) OOD distance, and (b) heterophilous nodes situated in homophilous graphs. In cases where these two conditions are met, fairness drops by up to 24% on real world datasets, and 30% in semi-synthetic datasets. Together, our theoretical insights, empirical analysis, and algorithmic contributions unveil a previously overlooked source of unfairness rooted in the graph's homophily information.
Paper Structure (26 sections, 1 theorem, 26 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 26 sections, 1 theorem, 26 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graphs
  4. Node Classification with GNNs
  5. Homophily and Heterophily
  6. Fairness
  7. Theoretical Relationship between Local Homophily and Fairness
  8. Theoretical Setup
  9. Impact of Out-of-Distribution Problem on Fairness
  10. New Real World Datasets
  11. Limitations of Current Datasets
  12. Proposed Real-World Datasets
  13. Semi-Synthetic Data Generation
  14. Empirical Analysis
  15. Train/Val/Test Processing
  16. ...and 11 more sections

Key Result

Theorem 3.1

Consider test nodes $u$ and $v$ with local homophily ratios $h + \alpha$, labels $y_u = 0$ and $y_v = 0$, and sensitive attributes $s_u = 0$ and $s_v = 1$. The difference in their expected logit associated with the correct label, $\mathbb{E}[\mathbf{p}_{u}]_{y_{u}}$ and $\mathbb{E}[\mathbf{p}_{v}]_{

Figures (4)

  • Figure 1: Local Homophily Distributions for Proposed Datasets. Each dataset displays a wide array of local and global homophily levels.
  • Figure 2: Comparison of Original, Generated, and Goal Distributions for Tolokers-Fair Dataset. Red and purple (no hatching) denote the original and modified distributions, respectively. The green (hatching) denotes the goal distribution. Overlap between the distributions indicates better re-wiring.
  • Figure 3: Example Train/Test Splits with differing $\boldsymbol{\gamma}$. Gray bars denote the $P_{G}$ for the Tolokers-Fair dataset. As $\gamma$ increases, the train and test sets become more disjoint, leading to more OOD samples in the test set. When $\gamma \approx 0$, train and test are identical.
  • Figure 4: Distribution of $\Delta$SP for Real and Semi-Synthetic Datasets. The histogram indicates that unadjusted GNNs (GCN, SAGE, and LINKX) produce more frequent SP increases compared to fairness-adjusted GNNs (Nifty and FairGNN).

Theorems & Definitions (6)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Definition 2.5
  • Theorem 3.1