A Benchmark for Fairness-Aware Graph Learning

TL;DR

This work tackles the lack of a unified benchmark for fairness-aware graph learning by designing a systematic evaluation protocol and benchmarking ten representative methods across seven real-world graphs, focusing on group and individual fairness as well as efficiency. It reveals distinct strengths: shallow embedding methods (e.g., FairWalk, CrossWalk) often excel on group-fairness metrics, while GNN-based methods provide stronger utility and broader individual-fairness coverage, albeit with higher computational costs. The study introduces practical guidance for practitioners, highlighting when to prioritize accuracy, fairness, or a balance of both, and which methods best align with specific fairness goals. Overall, the benchmark advances the ability to compare fairness-aware graph learning methods in a realistic setting, guiding future research and real-world deployment.

Abstract

Fairness-aware graph learning has gained increasing attention in recent years. Nevertheless, there lacks a comprehensive benchmark to evaluate and compare different fairness-aware graph learning methods, which blocks practitioners from choosing appropriate ones for broader real-world applications. In this paper, we present an extensive benchmark on ten representative fairness-aware graph learning methods. Specifically, we design a systematic evaluation protocol and conduct experiments on seven real-world datasets to evaluate these methods from multiple perspectives, including group fairness, individual fairness, the balance between different fairness criteria, and computational efficiency. Our in-depth analysis reveals key insights into the strengths and limitations of existing methods. Additionally, we provide practical guidance for applying fairness-aware graph learning methods in applications. To the best of our knowledge, this work serves as an initial step towards comprehensively understanding representative fairness-aware graph learning methods to facilitate future advancements in this area.

Figures (7)

• Figure 1: A timeline of the representative fairness-aware graph learning methods.
• Figure 2: Average rankings on AUC-ROC score and $\Delta_{\text{SP}}$ across all datasets. Methods are ranked in ascending order by the summation of two rankings.
• Figure 3: Pareto optimal frontier between AUC-ROC score and $\Delta_{\text{EO}}$ from FairGNN on Credit Default.
• Figure 4: Average rankings on $\Delta_{\text{SP}}$, $\Delta_{\text{EO}}$, and $\Delta_{\text{Utility}}$ across all datasets. Methods are ranked in ascending order by the summation of average rankings on all three fairness metrics.
• Figure 5: An exemplary comparison of AUC-ROC and running time across different collected graph learning methods on Credit Default dataset.