Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis

TL;DR

This work introduces the Contextualized Bi-Directional Dual Transformer (CBDT) for bias detection in text, pairing a Context Transformer for sentence-level bias with an Entity Transformer for token-level bias, both built on BERT. It presents a carefully constructed, FAIR-compliant corpus with bias dimensions and lexicons, annotated via multi-expert review, and evaluated across in-domain and out-of-distribution datasets. CBDT consistently outperforms traditional baselines and other transformers in both sequence and token classification tasks, while offering interpretable bias attributions through token-level attention. The study demonstrates practical applicability for bias auditing across domains and provides datasets and guidelines to promote fairness in NLP systems, while acknowledging language and modality limitations and outlining future multilingual and real-time debiasing directions.

Abstract

Bias detection in text is crucial for combating the spread of negative stereotypes, misinformation, and biased decision-making. Traditional language models frequently face challenges in generalizing beyond their training data and are typically designed for a single task, often focusing on bias detection at the sentence level. To address this, we present the Contextualized Bi-Directional Dual Transformer (CBDT) \textcolor{green}{\faLeaf} classifier. This model combines two complementary transformer networks: the Context Transformer and the Entity Transformer, with a focus on improving bias detection capabilities. We have prepared a dataset specifically for training these models to identify and locate biases in texts. Our evaluations across various datasets demonstrate CBDT \textcolor{green} effectiveness in distinguishing biased narratives from neutral ones and identifying specific biased terms. This work paves the way for applying the CBDT \textcolor{green} model in various linguistic and cultural contexts, enhancing its utility in bias detection efforts. We also make the annotated dataset available for research purposes.

Figures (3)

• Figure 1: Real-world example of bias, highlighting the need for NLP solutions.
• Figure 2: A visual representation of the CBDT method's workflow for bias detection in textual data.
• Figure 3: Performance trends of various classification models across training epochs.