Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Disentangling Dialect from Social Bias via Multitask Learning to Improve Fairness

Maximilian Spliethöver, Sai Nikhil Menon, Henning Wachsmuth

TL;DR

This work tackles dialect-induced unfairness in social bias detection by introducing a dialect-aware multitask framework. A shared encoder jointly learns five bias aspects and an auxiliary dialect task, trained with a round-robin cross-entropy objective over $k+1$ tasks and augmented by dialect labels attached to SBIC via a dialect classifier. Empirical results show state-of-the-art bias-detection performance and reduced dialect-based disparities, particularly for African-American English, with notable gains in equalized odds and predictive parity. The combination of multitask learning and dialect modeling improves both accuracy and fairness, and the approach can be extended to additional dialects and data-augmentation strategies to broaden applicability. Overall, the study provides evidence that encoding dialect patterns into models enhances fairness and robustness in bias detection across dialectal text.

Abstract

Dialects introduce syntactic and lexical variations in language that occur in regional or social groups. Most NLP methods are not sensitive to such variations. This may lead to unfair behavior of the methods, conveying negative bias towards dialect speakers. While previous work has studied dialect-related fairness for aspects like hate speech, other aspects of biased language, such as lewdness, remain fully unexplored. To fill this gap, we investigate performance disparities between dialects in the detection of five aspects of biased language and how to mitigate them. To alleviate bias, we present a multitask learning approach that models dialect language as an auxiliary task to incorporate syntactic and lexical variations. In our experiments with African-American English dialect, we provide empirical evidence that complementing common learning approaches with dialect modeling improves their fairness. Furthermore, the results suggest that multitask learning achieves state-of-the-art performance and helps to detect properties of biased language more reliably.

Disentangling Dialect from Social Bias via Multitask Learning to Improve Fairness

TL;DR

This work tackles dialect-induced unfairness in social bias detection by introducing a dialect-aware multitask framework. A shared encoder jointly learns five bias aspects and an auxiliary dialect task, trained with a round-robin cross-entropy objective over tasks and augmented by dialect labels attached to SBIC via a dialect classifier. Empirical results show state-of-the-art bias-detection performance and reduced dialect-based disparities, particularly for African-American English, with notable gains in equalized odds and predictive parity. The combination of multitask learning and dialect modeling improves both accuracy and fairness, and the approach can be extended to additional dialects and data-augmentation strategies to broaden applicability. Overall, the study provides evidence that encoding dialect patterns into models enhances fairness and robustness in bias detection across dialectal text.

Abstract

Dialects introduce syntactic and lexical variations in language that occur in regional or social groups. Most NLP methods are not sensitive to such variations. This may lead to unfair behavior of the methods, conveying negative bias towards dialect speakers. While previous work has studied dialect-related fairness for aspects like hate speech, other aspects of biased language, such as lewdness, remain fully unexplored. To fill this gap, we investigate performance disparities between dialects in the detection of five aspects of biased language and how to mitigate them. To alleviate bias, we present a multitask learning approach that models dialect language as an auxiliary task to incorporate syntactic and lexical variations. In our experiments with African-American English dialect, we provide empirical evidence that complementing common learning approaches with dialect modeling improves their fairness. Furthermore, the results suggest that multitask learning achieves state-of-the-art performance and helps to detect properties of biased language more reliably.
Paper Structure (47 sections, 1 equation, 2 figures, 20 tables)

This paper contains 47 sections, 1 equation, 2 figures, 20 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Joint Modeling of Social Bias and Dialect
  5. Data Augmentation with Dialect Labels
  6. Experiments
  7. Data
  8. TwitterAAE Corpus
  9. Social Bias Inference Corpus
  10. African-American English Classification
  11. Approach
  12. Baselines
  13. Measures
  14. Social Bias Detection
  15. Approach
  16. ...and 32 more sections

Figures (2)

  • Figure 1: Two texts from the corpus of sap2020, showcasing some of the five social bias aspects tackled in this paper: Neither text is lewd, talks about some target group, or is from an ingroup member. Unlike (a), however, (b) is offensive and intentional. While (a) contains elements common in AAE, i.e., the habitual be and dropped copula ziems2022, (b) does not.
  • Figure 2: Our joint learning architecture: Dialect classification is added as an additional head to the classification of the bias aspects. During training, all classification heads are trained round-robin in alternating manner. For inference, only the classification head of the primary task is used, here the Bias aspect $\#k$ head.