Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Topic-aware Causal Intervention for Counterfactual Detection

Thong Nguyen, Truc-My Nguyen

TL;DR

This work tackles CFD by identifying clue-phrase and label biases that limit generalization, especially under data imbalance. It proposes a topic-aware causal framework that integrates a neural topic representation with backdoor adjustment and deconfounded hidden representations, enabling robust CFD predictions across languages and bias-sensitive tasks. The approach achieves state-of-the-art results on CFD benchmarks (SemEval-2020, Amazon-2021) and demonstrates cross-lingual and transferability benefits to related tasks like PI and ISA, with extensive ablations and case studies supporting the effectiveness of causal intervention. The proposed deconfounding strategy, combining a topic-driven global semantics with causal intervention, has practical implications for fairer, more reliable NLP systems in multilingual and bias-prone settings.

Abstract

Counterfactual statements, which describe events that did not or cannot take place, are beneficial to numerous NLP applications. Hence, we consider the problem of counterfactual detection (CFD) and seek to enhance the CFD models. Previous models are reliant on clue phrases to predict counterfactuality, so they suffer from significant performance drop when clue phrase hints do not exist during testing. Moreover, these models tend to predict non-counterfactuals over counterfactuals. To address these issues, we propose to integrate neural topic model into the CFD model to capture the global semantics of the input statement. We continue to causally intervene the hidden representations of the CFD model to balance the effect of the class labels. Extensive experiments show that our approach outperforms previous state-of-the-art CFD and bias-resolving methods in both the CFD and other bias-sensitive tasks.

Topic-aware Causal Intervention for Counterfactual Detection

TL;DR

This work tackles CFD by identifying clue-phrase and label biases that limit generalization, especially under data imbalance. It proposes a topic-aware causal framework that integrates a neural topic representation with backdoor adjustment and deconfounded hidden representations, enabling robust CFD predictions across languages and bias-sensitive tasks. The approach achieves state-of-the-art results on CFD benchmarks (SemEval-2020, Amazon-2021) and demonstrates cross-lingual and transferability benefits to related tasks like PI and ISA, with extensive ablations and case studies supporting the effectiveness of causal intervention. The proposed deconfounding strategy, combining a topic-driven global semantics with causal intervention, has practical implications for fairer, more reliable NLP systems in multilingual and bias-prone settings.

Abstract

Counterfactual statements, which describe events that did not or cannot take place, are beneficial to numerous NLP applications. Hence, we consider the problem of counterfactual detection (CFD) and seek to enhance the CFD models. Previous models are reliant on clue phrases to predict counterfactuality, so they suffer from significant performance drop when clue phrase hints do not exist during testing. Moreover, these models tend to predict non-counterfactuals over counterfactuals. To address these issues, we propose to integrate neural topic model into the CFD model to capture the global semantics of the input statement. We continue to causally intervene the hidden representations of the CFD model to balance the effect of the class labels. Extensive experiments show that our approach outperforms previous state-of-the-art CFD and bias-resolving methods in both the CFD and other bias-sensitive tasks.
Paper Structure (24 sections, 1 theorem, 14 equations, 10 figures, 10 tables)

This paper contains 24 sections, 1 theorem, 14 equations, 10 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Representational Intervention for Deep Learning
  4. Predictive Biases in Deep Learning
  5. Methodology
  6. Problem Formulation
  7. Neural Topic Model (NTM)
  8. Causal Perspective into Counterfactual Detection
  9. Model Implementation
  10. Training Strategy
  11. Experiments
  12. Datasets
  13. Implementation Details
  14. Baselines
  15. Comparison with State-of-the-arts
  16. ...and 9 more sections

Key Result

Theorem 1

(Backdoor Adjustment pearl2009causal) Let $o \in \{y, \boldsymbol{\theta}\}$, $i \in \{\mathbf{x}_{\textup{bow}}, \mathbf{h}\}$, and $n \in \{d_{\textup{TM}}, d_{\textup{CFD}}\}$. Then,

Figures (10)

  • Figure 1: For each topic, we count the percentage of inputs in which the topic has the largest probability in the topic representation. Topic $1$, $2$, and $9$ refer to three top topics of the input document, in descending order of probability.
  • Figure 2: (left) Our proposed causal model for counterfactual detection. (right) The causal graph after removing arrows from $D_{\text{CFD}}$ to $H$ and $D_{\text{TM}}$ to $X_{\text{bow}}$, eliminating spurious effects of the label and topic biases.
  • Figure 3: Illustration of the Topic-aware Causal Intervention Framework for Counterfactual Detection. Here the green component denotes the neural topic model, the purple component the text encoder, and the orange component our causal intervention operation for counterfactuality prediction.
  • Figure 4: Attention weights of the [CLS] token to all other words, and output scores of mBERT and Our Model. The score is in range $[0, 1]$. The input: "The girlfriend was annoying, and it made me wonder if any man in his right mind would have put up with her behavior as long as he did."
  • Figure 5: Topic Percentages and inferred Top Topics from Figure \ref{['fig:topic_distribution']} after Causal Intervention.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Theorem 1