Fair CoVariance Neural Networks

TL;DR

FVNNs address biases encoded in covariance matrices by combining fair covariance estimation with end-to-end fairness penalties in a covariance-based graph neural framework. They demonstrate that stability of covariance neural networks yields fairer predictions than fair PCA, especially in low-sample regimes, and allow tuning the fairness-accuracy tradeoff via a penalty weight. Empirical results on synthetic data and real-world regression and classification tasks show improved fairness with flexible covariate debiasing options and robust performance. The work highlights the potential of covariance-based learning with fairness regularization for reliable, group-aware predictions in biased data settings.

Abstract

Covariance-based data processing is widespread across signal processing and machine learning applications due to its ability to model data interconnectivities and dependencies. However, harmful biases in the data may become encoded in the sample covariance matrix and cause data-driven methods to treat different subpopulations unfairly. Existing works such as fair principal component analysis (PCA) mitigate these effects, but remain unstable in low sample regimes, which in turn may jeopardize the fairness goal. To address both biases and instability, we propose Fair coVariance Neural Networks (FVNNs), which perform graph convolutions on the covariance matrix for both fair and accurate predictions. Our FVNNs provide a flexible model compatible with several existing bias mitigation techniques. In particular, FVNNs allow for mitigating the bias in two ways: first, they operate on fair covariance estimates that remove biases from their principal components; second, they are trained in an end-to-end fashion via a fairness regularizer in the loss function so that the model parameters are tailored to solve the task directly in a fair manner. We prove that FVNNs are intrinsically fairer than analogous PCA approaches thanks to their stability in low sample regimes. We validate the robustness and fairness of our model on synthetic and real-world data, showcasing the flexibility of FVNNs along with the tradeoff between fair and accurate performance.

Figures (2)

• Figure 1: Performance of PCA-based and FVNN models for a synthetic regression task. Each model is compared using the sample covariance ${\hat{\mathbf C} }$ and the balanced covariance $\hat{{\mathbf C}}_\mathrm{bal}$. (a) Fairness measured as imbalance in sMAPE across groups. (b) Error measured as sMAPE. The legend is shared by both plots.
• Figure 2: Performance of PCA-based and FVNN models for real-world regression and classification tasks. For each plot, the $y$-axis denotes bias and the $x$-axis error. (a) Parkinson regression as the bias penalty weight $\gamma$ increases. Results for PCA with $\hat{{\mathbf C}}_\mathrm{deb}$ are overlapped with those with ${\hat{\mathbf C} }$, which are therefore not visible. (b) LSAC regression as the bias penalty weight $\gamma$ increases. (c) German Credit classification. FVNN is shown with and without a bias penalty, while PCA-based models are shown with 10 and 30 PCs. The legend is shared by all three plots.