Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection

Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, Yoav Goldberg

TL;DR

The paper introduces Iterative Null-space Projection (INLP), a data-driven method to remove linear information about protected attributes from neural representations by iteratively training linear classifiers and projecting data onto their nullspaces. By constructing a guard through successive nullspace projections, INLP yields a final transformation that makes protected attributes linearly undetectable while preserving end-task information to varying degrees. The authors demonstrate INLP on debiasing word embeddings and improving fairness in classification in both controlled and real-world settings, and provide theoretical guarantees about the guarding subspace and the minimal distortion of representation geometry. Limitations include dependence on data distribution and the restriction to linear (not non-linear) leakage of protected information, with future work exploring broader use-cases such as style transfer and disentanglement.

Abstract

The ability to control for the kinds of information encoded in neural representation has a variety of use cases, especially in light of the challenge of interpreting these models. We present Iterative Null-space Projection (INLP), a novel method for removing information from neural representations. Our method is based on repeated training of linear classifiers that predict a certain property we aim to remove, followed by projection of the representations on their null-space. By doing so, the classifiers become oblivious to that target property, making it hard to linearly separate the data according to it. While applicable for multiple uses, we evaluate our method on bias and fairness use-cases, and show that our method is able to mitigate bias in word embeddings, as well as to increase fairness in a setting of multi-class classification.

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection

TL;DR

The paper introduces Iterative Null-space Projection (INLP), a data-driven method to remove linear information about protected attributes from neural representations by iteratively training linear classifiers and projecting data onto their nullspaces. By constructing a guard through successive nullspace projections, INLP yields a final transformation that makes protected attributes linearly undetectable while preserving end-task information to varying degrees. The authors demonstrate INLP on debiasing word embeddings and improving fairness in classification in both controlled and real-world settings, and provide theoretical guarantees about the guarding subspace and the minimal distortion of representation geometry. Limitations include dependence on data distribution and the restriction to linear (not non-linear) leakage of protected information, with future work exploring broader use-cases such as style transfer and disentanglement.

Abstract

The ability to control for the kinds of information encoded in neural representation has a variety of use cases, especially in light of the challenge of interpreting these models. We present Iterative Null-space Projection (INLP), a novel method for removing information from neural representations. Our method is based on repeated training of linear classifiers that predict a certain property we aim to remove, followed by projection of the representations on their null-space. By doing so, the classifiers become oblivious to that target property, making it hard to linearly separate the data according to it. While applicable for multiple uses, we evaluate our method on bias and fairness use-cases, and show that our method is able to mitigate bias in word embeddings, as well as to increase fairness in a setting of multi-class classification.

Paper Structure

This paper contains 32 sections, 5 theorems, 16 equations, 6 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Objective and Definitions
  4. Guarded w.r.t. a hypothesis class
  5. Guarding function
  6. Iterative Nullspace Projection
  7. Nullspace Projection
  8. Iterative Projection
  9. Implementation Details
  10. Application to Fair Classification
  11. Experiments and Analysis
  12. "Debiasing" Word Embeddings
  13. Data.
  14. Procedure
  15. Results
  16. ...and 17 more sections

Key Result

Lemma A.1

if $w_2$ is initialized as the zero vector and trained with SGD, and the loss $\mathcal{L}$ is convex, then $w_2$ is orthogonal to $w_1$, that is, $w_1 \cdot w_2 = 0$.

Figures (6)

  • Figure 1: t-SNE projection of GloVe vectors of the most gender-biased words after t=0, 3, 18, and 35 iterations of INLP. Words are colored according to being male-biased or female-biased.
  • Figure 1:
  • Figure 2: Nullspace projection for a 2-dimensional binary classifier. The decision boundary of $W$ is $W$ 's null-space.
  • Figure 2: The relative change of biased vs. random words, per profession.
  • Figure 3: t-SNE projection of BERT representations for the profession "professor" (left) and for a random sample of all professions (right), before and after the projection.
  • ...and 1 more figures

Theorems & Definitions (10)

  • Lemma A.1
  • proof
  • Corollary A.1.1
  • proof
  • Corollary A.1.2
  • proof
  • Corollary A.1.3
  • proof
  • Lemma A.2
  • proof