From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling

TL;DR

This study investigates how bias propagates in language models across architecture (n-gram vs. transformer) and data history, using a comparative behavioral framework. It jointly analyzes data provenance, bias injection, and architectural factors, with CrowS-Pairs as the evaluation benchmark. Key findings show that n-gram bias depends on context window and smoothing (Modified Kneser-Ney mitigates bias effectively), while transformers exhibit architectural robustness to depth, heads, and attention type, though bias remains sensitive to data provenance and injected bias. Temporal effects reveal nontrivial shifts in bias across Wikipedia snapshots, and bias amplification is category-dependent, with sexual orientation consistently highly amplified and nationality relatively muted. The results advocate for holistic bias mitigation that traces origins in both data and architecture, rather than addressing symptoms alone, to improve the safety and fairness of language models in real-world deployment.

Abstract

Current research on bias in language models (LMs) predominantly focuses on data quality, with significantly less attention paid to model architecture and temporal influences of data. Even more critically, few studies systematically investigate the origins of bias. We propose a methodology grounded in comparative behavioral theory to interpret the complex interaction between training data and model architecture in bias propagation during language modeling. Building on recent work that relates transformers to n-gram LMs, we evaluate how data, model design choices, and temporal dynamics affect bias propagation. Our findings reveal that: (1) n-gram LMs are highly sensitive to context window size in bias propagation, while transformers demonstrate architectural robustness; (2) the temporal provenance of training data significantly affects bias; and (3) different model architectures respond differentially to controlled bias injection, with certain biases (e.g. sexual orientation) being disproportionately amplified. As language models become ubiquitous, our findings highlight the need for a holistic approach -- tracing bias to its origins across both data and model dimensions, not just symptoms, to mitigate harm.

Figures (12)

• Figure 1: Violin plots comparing bias score distribution between $n$-gram and transformer models across Wikipedia dumps (2018–2024) for different bias injection levels (0%, 33%, 100%). The white dot indicates the median, the thick bar shows the interquartile range (IQR). The kernel density estimate (KDE) reveals the full score distribution.
• Figure 2: Effect of controlled bias injection on model bias scores for $n$-gram and transformer LMs. Each figure shows the distribution of bias scores across different levels of synthetic bias injection ($0\%$, $33\%$, and $100\%$) into the Wikipedia data. Individual data points represent bias scores for each model configuration, boxplots summarize the distribution, and the red dashed line indicates the regression-predicted mean bias score for each injection level.
• Figure 3: Pairwise comparisons of bias amplification across categories using t-tests with Bonferroni correction. The lower triangle shows t-statistics (row vs. column) with significance markers: $*p < 0.05$, $**p < 0.01$, $***p < 0.001$. Positive $t$-values reflect amplified biases (e.g., sexual orientation), negative values indicate suppressed biases (e.g., nationality), with magnitude showing effect strength varied by color intensities.
• Figure 4: Mean bias scores for each bias type aggregated across years. The bar heights represent the average bias score for each category, as measured by the CrowS-Pairs dataset.
• Figure 5: Injected bias sensitivity across different data scales and model types.