Tracing Stereotypes in Pre-trained Transformers: From Biased Neurons to Fairer Models

TL;DR

The paper addresses how social biases are internalized in pre-trained transformers used in software engineering tasks and asks whether stereotypical associations reside in small neuron subsets. It adapts the knowledge-neuron framework to define biased relations, builds a dataset of 1,018 biased relations and 10,180 bias-activating prompts, and applies integrated gradients to locate biased neurons. By erasing these neurons, the study finds significant increases in perplexity for bias-related prompts but minimal impact on control prompts, indicating a causal and localized effect on stereotype expression. Across five SE tasks, neuron suppression yields limited performance degradation and, in some cases, improved fluency (notably in masked language modeling), suggesting a practical path to fairness that preserves utility, especially for task-adapted models.

Abstract

The advent of transformer-based language models has reshaped how AI systems process and generate text. In software engineering (SE), these models now support diverse activities, accelerating automation and decision-making. Yet, evidence shows that these models can reproduce or amplify social biases, raising fairness concerns. Recent work on neuron editing has shown that internal activations in pre-trained transformers can be traced and modified to alter model behavior. Building on the concept of knowledge neurons, neurons that encode factual information, we hypothesize the existence of biased neurons that capture stereotypical associations within pre-trained transformers. To test this hypothesis, we build a dataset of biased relations, i.e., triplets encoding stereotypes across nine bias types, and adapt neuron attribution strategies to trace and suppress biased neurons in BERT models. We then assess the impact of suppression on SE tasks. Our findings show that biased knowledge is localized within small neuron subsets, and suppressing them substantially reduces bias with minimal performance loss. This demonstrates that bias in transformers can be traced and mitigated at the neuron level, offering an interpretable approach to fairness in SE.

Figures (3)

• Figure 1: Overview of the Research Method Proposed.
• Figure 2: Average perplexity (PPL) increase ratio after biased neuron suppression across the nine biased relations.
• Figure 3: Correlation between the number of suppressed biased neurons and perplexity increase ratio across relations.