Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Assessing the quality and coherence of word embeddings after SCM-based intersectional bias mitigation

Eren Kocadag, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansoor Alsahag

TL;DR

The paper extends SCM-guided debiasing to intersectional bias in static word embeddings by constructing compound representations for gender-race pairs via Summation or Concatenation and projecting them into a two-dimensional SCM subspace derived by PCA. It compares three debiasing operators—Subtraction, Linear Projection, and Partial Projection—across Word2Vec, GloVe, and ConceptNet Numberbatch, evaluating neighborhood coherence with Embedding Coherence Test (ECT) and analogy preservation with Embedding Quality Test (EQT). Results show that intersectional SCM debiasing largely preserves the semantic landscape, with a predictable trade-off: methods tightly preserving geometry can dampen analogies, while more assertive projections can improve analogies at some neighborhood cost. The study provides practical guidance on aggregation and debiasing choices to balance stability and analogy performance, demonstrating that intersectional SCM debiasing is feasible for static embeddings and can be tailored to embedding family and application needs.

Abstract

Static word embeddings often absorb social biases from the text they learn from, and those biases can quietly shape downstream systems. Prior work that uses the Stereotype Content Model (SCM) has focused mostly on single-group bias along warmth and competence. We broaden that lens to intersectional bias by building compound representations for pairs of social identities through summation or concatenation, and by applying three debiasing strategies: Subtraction, Linear Projection, and Partial Projection. We study three widely used embedding families (Word2Vec, GloVe, and ConceptNet Numberbatch) and assess them with two complementary views of utility: whether local neighborhoods remain coherent and whether analogy behavior is preserved. Across models, SCM-based mitigation carries over well to the intersectional case and largely keeps the overall semantic landscape intact. The main cost is a familiar trade off: methods that most tightly preserve geometry tend to be more cautious about analogy behavior, while more assertive projections can improve analogies at the expense of strict neighborhood stability. Partial Projection is reliably conservative and keeps representations steady; Linear Projection can be more assertive; Subtraction is a simple baseline that remains competitive. The choice between summation and concatenation depends on the embedding family and the application goal. Together, these findings suggest that intersectional debiasing with SCM is practical in static embeddings, and they offer guidance for selecting aggregation and debiasing settings when balancing stability against analogy performance.

Assessing the quality and coherence of word embeddings after SCM-based intersectional bias mitigation

TL;DR

The paper extends SCM-guided debiasing to intersectional bias in static word embeddings by constructing compound representations for gender-race pairs via Summation or Concatenation and projecting them into a two-dimensional SCM subspace derived by PCA. It compares three debiasing operators—Subtraction, Linear Projection, and Partial Projection—across Word2Vec, GloVe, and ConceptNet Numberbatch, evaluating neighborhood coherence with Embedding Coherence Test (ECT) and analogy preservation with Embedding Quality Test (EQT). Results show that intersectional SCM debiasing largely preserves the semantic landscape, with a predictable trade-off: methods tightly preserving geometry can dampen analogies, while more assertive projections can improve analogies at some neighborhood cost. The study provides practical guidance on aggregation and debiasing choices to balance stability and analogy performance, demonstrating that intersectional SCM debiasing is feasible for static embeddings and can be tailored to embedding family and application needs.

Abstract

Static word embeddings often absorb social biases from the text they learn from, and those biases can quietly shape downstream systems. Prior work that uses the Stereotype Content Model (SCM) has focused mostly on single-group bias along warmth and competence. We broaden that lens to intersectional bias by building compound representations for pairs of social identities through summation or concatenation, and by applying three debiasing strategies: Subtraction, Linear Projection, and Partial Projection. We study three widely used embedding families (Word2Vec, GloVe, and ConceptNet Numberbatch) and assess them with two complementary views of utility: whether local neighborhoods remain coherent and whether analogy behavior is preserved. Across models, SCM-based mitigation carries over well to the intersectional case and largely keeps the overall semantic landscape intact. The main cost is a familiar trade off: methods that most tightly preserve geometry tend to be more cautious about analogy behavior, while more assertive projections can improve analogies at the expense of strict neighborhood stability. Partial Projection is reliably conservative and keeps representations steady; Linear Projection can be more assertive; Subtraction is a simple baseline that remains competitive. The choice between summation and concatenation depends on the embedding family and the application goal. Together, these findings suggest that intersectional debiasing with SCM is practical in static embeddings, and they offer guidance for selecting aggregation and debiasing settings when balancing stability against analogy performance.
Paper Structure (18 sections, 10 equations, 3 figures, 10 tables)

This paper contains 18 sections, 10 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data and Problem Setting
  4. Method
  5. Word embeddings
  6. Word pairs
  7. Defining the bias subspace
  8. Bias mitigation methods
  9. Evaluation
  10. Data and Problem Setting
  11. Results
  12. Ablations and Analysis
  13. Ablation A1: Effect of the aggregation strategy.
  14. Ablation A2: Effect of the debiasing method.
  15. Ablation A3: Single group versus intersectional targets.
  16. ...and 3 more sections

Figures (3)

  • Figure 1: An overview of the methodology for bias mitigation and for evaluating the coherence and quality of the word embeddings.
  • Figure 2: The original terms used by Omrani et al. Omrani2023, with the race-related terms on the left side and the gender-related terms on the right.
  • Figure 3: The terms used for bias mitigation in this study inspired by Choenni et al. Choenni2021. The words in the first list are related to gender, while the words in the second list are related to race.