Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis

TL;DR

The paper investigates how pretrained language models exploit spurious correlations in text classification by revealing misalignment of spurious and genuine tokens in embedding space through neighborhood analysis. It introduces NFL, a family of regularization methods that preserve token semantics by restricting updates to model outputs or parameters, achieving substantial robustness gains without requiring auxiliary unbiased data. The authors define a spurious token score based on changes to token neighbors and demonstrate the approach on sentiment and toxicity tasks across multiple PLMs, including cases with naturally occurring spurious correlations. The work offers a practical path to more robust NLP systems through neighborhood-informed analysis and regularization, with code and experiments indicating near-ideal out-of-distribution performance under bias.

Abstract

Recent research has revealed that machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. For instance, a sentiment classifier may erroneously learn that the token "performances" is commonly associated with positive movie reviews. Relying on these spurious correlations degrades the classifiers performance when it deploys on out-of-distribution data. In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis. The analysis uncovers how spurious correlations lead unrelated words to erroneously cluster together in the embedding space. Driven by the analysis, we design a metric to detect spurious tokens and also propose a family of regularization methods, NFL (doN't Forget your Language) to mitigate spurious correlations in text classification. Experiments show that NFL can effectively prevent erroneous clusters and significantly improve the robustness of classifiers without auxiliary data. The code is publicly available at https://github.com/oscarchew/doNt-Forget-your-Language.

Figures (6)

• Figure 1: t-SNE projections of representations before and after fine-tuning. book, movie erroneously align with genuine positive, negative tokens respectively after fine-tuning, preventing the classifier from distinguishing between spurious and genuine tokens.
• Figure 2: Comparison of fine-tuning and NFL. Red and blue regions represent trainable and frozen parameters respectively. Standard fine-tuning: every parameter is trainable; NFL-F: only the classification head is trainable; NFL-PT: the continuous prompts and the classification head are trainable; NFL-CO/NFL-CP: every parameter is trainable but changes in the language model are restricted by the regularization term in the loss function.
• Figure 3: t-SNE projections of representations after fine-tuning with NFL-CO/NFL-CP. By preventing the formation of erroneous clusters, NFL learns robust representations.
• Figure 4: Amazon binary results with different PLMs. Blue bars represent robust accuracies and red bars represent robustness gaps. The robustness gaps are smaller in PLMs with better initial representations.
• Figure 5: t-SNE projections of representations after fine-tuning on data with bias ratios of $0.8$ (top) and $0.9$ (bottom).