Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Extending Fair Null-Space Projections for Continuous Attributes to Kernel Methods

Felix Störck, Fabian Hinder, Barbara Hammer

TL;DR

The paper tackles fairness in regression with continuous protected attributes by extending the fair null-space projection approach to kernel methods. It introduces a kernel-level transformation that removes information predictive of protected attributes within the empirical feature space, while preserving kernel structure and enabling out-of-sample application. The method is model-agnostic and demonstrated through SVR and KRR variants (FKD), showing competitive or improved performance against contemporary baselines across real-world datasets and under multiple protected attributes. The work also discusses computational strategies, notably Nyström approximations, to scale the approach and outlines limitations and avenues for future research in broader kernel methods and privacy-aware applications.

Abstract

With the on-going integration of machine learning systems into the everyday social life of millions the notion of fairness becomes an ever increasing priority in their development. Fairness notions commonly rely on protected attributes to assess potential biases. Here, the majority of literature focuses on discrete setups regarding both target and protected attributes. The literature on continuous attributes especially in conjunction with regression -- we refer to this as \emph{continuous fairness} -- is scarce. A common strategy is iterative null-space projection which as of now has only been explored for linear models or embeddings such as obtained by a non-linear encoder. We improve on this by generalizing to kernel methods, significantly extending the scope. This yields a model and fairness-score agnostic method for kernel embeddings applicable to continuous protected attributes. We demonstrate that our novel approach in conjunction with Support Vector Regression (SVR) provides competitive or improved performance across multiple datasets in comparisons to other contemporary methods.

Extending Fair Null-Space Projections for Continuous Attributes to Kernel Methods

TL;DR

The paper tackles fairness in regression with continuous protected attributes by extending the fair null-space projection approach to kernel methods. It introduces a kernel-level transformation that removes information predictive of protected attributes within the empirical feature space, while preserving kernel structure and enabling out-of-sample application. The method is model-agnostic and demonstrated through SVR and KRR variants (FKD), showing competitive or improved performance against contemporary baselines across real-world datasets and under multiple protected attributes. The work also discusses computational strategies, notably Nyström approximations, to scale the approach and outlines limitations and avenues for future research in broader kernel methods and privacy-aware applications.

Abstract

With the on-going integration of machine learning systems into the everyday social life of millions the notion of fairness becomes an ever increasing priority in their development. Fairness notions commonly rely on protected attributes to assess potential biases. Here, the majority of literature focuses on discrete setups regarding both target and protected attributes. The literature on continuous attributes especially in conjunction with regression -- we refer to this as \emph{continuous fairness} -- is scarce. A common strategy is iterative null-space projection which as of now has only been explored for linear models or embeddings such as obtained by a non-linear encoder. We improve on this by generalizing to kernel methods, significantly extending the scope. This yields a model and fairness-score agnostic method for kernel embeddings applicable to continuous protected attributes. We demonstrate that our novel approach in conjunction with Support Vector Regression (SVR) provides competitive or improved performance across multiple datasets in comparisons to other contemporary methods.

Paper Structure

This paper contains 31 sections, 3 theorems, 23 equations, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fairness in Machine Learning
  4. Continuous Fairness Measures
  5. Continuous Fairness: Social Implications
  6. Sub-space Projections for Fairness
  7. Fairness in Kernel Methods
  8. Method
  9. Motivation: Feature Space
  10. Fairness in the Empirical Feature Space
  11. Choice of Algorithm
  12. Computational Complexity and Nystroem
  13. Experimental Results
  14. Real-world Datasets
  15. Methods
  16. ...and 16 more sections

Key Result

Lemma 3.1

$\prod_{i=0}^{m-1} \mathbf{P}^{\mathbf{G}_{(i)}}$ is a projection.

Figures (3)

  • Figure 1: Pareto fronts for the MAE (y-axis, lower is better) and three different fairness measures (x-axis, lower is better) reported with empirical standard deviations (error-bars). Comparison for multiple methods (marker, color) and datasets (boxes, one per row). Number of iterations $m$ for *-FKD, regularization $\mu$ for *-FKL and HGR penalty $\lambda$ are found in \ref{['sec:add_exp_det']}.
  • Figure 2: Other experiments on the "Crimes" dataset. (a): Performance (MAE, y-axis, lower is better) of our "SVR-FKD" considering both the white and black population percentage (pop. perc.) as protected (multi, squares, cyan, $m\in\{0, 5, 12, 25, 35, 45\}$) and only the black pop. perc. (single, circles, orange, $m\in\{0, 10, 35, 50, 70, 85\}$). GDP fairness (x-axis, lower is better) with respect to the black pop. perc. (top) and to the white pop. perc. (bottom) (b): Influence of $\tilde{\alpha}$ on the performance (MAE, y-axis, lower is better) of our "KRR-FKD" for fixed parameters and iterations $m\in\{3, 5\}$ (color and marker). GDP fairness (x-axis, lower is better) w.r.t the black pop. perc.
  • Figure 3: Nystroem approximation with different percentages of components for "SVR-FKD". "Crimes" dataset with the same experimental setting as before.

Theorems & Definitions (7)

  • Lemma 3.1
  • proof
  • Theorem 3.2
  • Corollary 3.3
  • proof
  • proof
  • proof