Explanation based Bias Decoupling Regularization for Natural Language Inference
Jianxiang Zang, Hui Liu
TL;DR
The paper tackles robustness in Natural Language Inference by shifting from sample-level debiasing to identifying biased parts within samples through human explanations. It introduces Explanation based Bias Decoupling Regularization (EBD-Reg), which uses three parallel supervision targets—Distinguishing, Decoupling, Aligning—and Adaptive Token-level Attention to emphasize keywords while suppressing biases, guided by explanations from the eSNLI dataset. The training objective combines main and auxiliary losses $\mathcal{L}=\mathcal{L}_{\text{main}}+\alpha\mathcal{L}_{ER}+\frac{\beta}{H}\sum_{h=1}^H(\mathcal{L}^h_{SA}+\mathcal{L}^h_{SI})$ with $\alpha=0.4$, $\beta=0.8$, $H=3$, and demonstrates improved out-of-distribution performance across SNLI, MNLI, RTE, QNLI, ANLI, SciTail, and HANS on multiple Transformer backbones. Empirical results show strong gains from combining EBD-Reg with ATA, with ablations highlighting the complementary roles of keyword-vs-bias distinction, self-attention guidance, and sub-inference alignment. The approach provides a practical, interpretable route to debias NLI models and suggests broader applicability to other reasoning tasks where human explanations can identify causal features.
Abstract
The robustness of Transformer-based Natural Language Inference encoders is frequently compromised as they tend to rely more on dataset biases than on the intended task-relevant features. Recent studies have attempted to mitigate this by reducing the weight of biased samples during the training process. However, these debiasing methods primarily focus on identifying which samples are biased without explicitly determining the biased components within each case. This limitation restricts those methods' capability in out-of-distribution inference. To address this issue, we aim to train models to adopt the logic humans use in explaining causality. We propose a simple, comprehensive, and interpretable method: Explanation based Bias Decoupling Regularization (EBD-Reg). EBD-Reg employs human explanations as criteria, guiding the encoder to establish a tripartite parallel supervision of Distinguishing, Decoupling and Aligning. This method enables encoders to identify and focus on keywords that represent the task-relevant features during inference, while discarding the residual elements acting as biases. Empirical evidence underscores that EBD-Reg effectively guides various Transformer-based encoders to decouple biases through a human-centric lens, significantly surpassing other methods in terms of out-of-distribution inference capabilities.