Endowing Pre-trained Graph Models with Provable Fairness

TL;DR

This work addresses fairness in pre-trained graph models by freezing the PGM and introducing GraphPAR, a parameter-efficient adapter-tuning framework. It leverages a sensitive semantic augmenter to extend node representations, then trains an adapter via RandAT or MinMax to remove dependence on sensitive semantics, and finally provides provable fairness through center and random smoothing (guarantees that d_cs < d_rs). Empirical results on real-world datasets show GraphPAR achieves state-of-the-art prediction performance while significantly improving fairness, with around 90% of nodes attaining provable fairness. The approach decouples debiasing from PGM fine-tuning, offering a flexible, theoretically grounded, and resource-efficient solution for fair graph learning.

Abstract

Pre-trained graph models (PGMs) aim to capture transferable inherent structural properties and apply them to different downstream tasks. Similar to pre-trained language models, PGMs also inherit biases from human society, resulting in discriminatory behavior in downstream applications. The debiasing process of existing fair methods is generally coupled with parameter optimization of GNNs. However, different downstream tasks may be associated with different sensitive attributes in reality, directly employing existing methods to improve the fairness of PGMs is inflexible and inefficient. Moreover, most of them lack a theoretical guarantee, i.e., provable lower bounds on the fairness of model predictions, which directly provides assurance in a practical scenario. To overcome these limitations, we propose a novel adapter-tuning framework that endows pre-trained graph models with provable fairness (called GraphPAR). GraphPAR freezes the parameters of PGMs and trains a parameter-efficient adapter to flexibly improve the fairness of PGMs in downstream tasks. Specifically, we design a sensitive semantic augmenter on node representations, to extend the node representations with different sensitive attribute semantics for each node. The extended representations will be used to further train an adapter, to prevent the propagation of sensitive attribute semantics from PGMs to task predictions. Furthermore, with GraphPAR, we quantify whether the fairness of each node is provable, i.e., predictions are always fair within a certain range of sensitive attribute semantics. Experimental evaluations on real-world datasets demonstrate that GraphPAR achieves state-of-the-art prediction performance and fairness on node classification task. Furthermore, based on our GraphPAR, around 90\% nodes have provable fairness.

Figures (12)

• Figure 1: An example of evaluating the node classification fairness of PGMs on Pokec_z and Pokec_n datasets. DP (↓) and EO (↓) report the fairness of three PGMs (i.e., DGI, EdgePred, GCA) and vanilla GCN.
• Figure 2: Overview of GraphPAR. In the adapter tuning phase, we first utilize the PGMs to obtain node representations $\mathbf{H}$. Then, we design a sensitive semantic augmenter to extend the node representations with different sensitive attribute semantics, i.e., sensitive attribute samples $\mathcal{S}$. Finally, the extended node representations are used to train an adapter, transforming the node representations to be independent of sensitive attribute semantics. In the provable fairness phase, based on the smoothed versions of the well-trained adapter and classifier, we use the smooth adapter to get its output bound guarantee $d_{cs}$ and use the smooth classifier to get its local robustness guarantee $d_{rs}$. Sequentially, we quantify whether the fairness of each node is provable by comparing $d_{cs}$ with $d_{rs}$.
• Figure 3: Sensitive attribute prediction accuracy under different augmentation degree $t$.
• Figure 4: The effect of augmentation degree $t$ to node representations on pokec_z.
• Figure 5: The effect of augmentation range $\epsilon$ to $\text{GraphPAR}_{minmax}$ and $\text{GraphPAR}_{RandAT}$ on Pokec_z.

Theorems & Definitions (6)

• Definition 1: Fairness on Graph
• Definition 2: Fairness of PGMs
• Theorem 1: Center Smoothing kumar2021center
• Theorem 2
• Definition 3: Provable Fairness of PGMs
• Theorem 3