Explanation Regularisation through the Lens of Attributions

TL;DR

This work critically examines Explanation Regularisation (ER) and its claimed boost to out-of-domain robustness by steering models toward human-plausible tokens. By systematically analyzing both local and global attribution-guided training, including constrained optimisation, the authors show that increases in plausibility do not reliably imply greater reliance on plausible features, and that OOD improvements are not predicted by in-domain plausibility. Local guidance can be exploited to lower the explanation loss without genuine changes in attribution across layers, while global guidance only meaningfully shifts attributions under tight bounds, often at the cost of classification accuracy. The findings challenge the central assumption of ER’s effectiveness for OOD robustness and suggest that future work should carefully balance plausibility constraints with overall predictive performance, possibly via selective regularisation of specific layers or heads and by exploring more informative global guidance strategies.

Abstract

Explanation regularisation (ER) has been introduced as a way to guide text classifiers to form their predictions relying on input tokens that humans consider plausible. This is achieved by introducing an auxiliary explanation loss that measures how well the output of an input attribution technique for the model agrees with human-annotated rationales. The guidance appears to benefit performance in out-of-domain (OOD) settings, presumably due to an increased reliance on "plausible" tokens. However, previous work has under-explored the impact of guidance on that reliance, particularly when reliance is measured using attribution techniques different from those used to guide the model. In this work, we seek to close this gap, and also explore the relationship between reliance on plausible features and OOD performance. We find that the connection between ER and the ability of a classifier to rely on plausible features has been overstated and that a stronger reliance on plausible tokens does not seem to be the cause for OOD improvements.

Figures (14)

• Figure 1: ER (top-left) minimises a classification and an explanation loss. The latter uses an input attribution technique to obtain a machine rationale for the prediction, and penalises differences between that and the human rationale. Guiding the model to rely in its predictions on plausible tokens is expected to help the classifier at test time (top-right), when no human rationale is available. Bottom: even though the attribution technique used for guidance shows human-like rationales, the model may in fact rely on different (non-plausible) tokens, as other attribution techniques might reveal.
• Figure 2: F1-Macro scores ($\uparrow$). ER + Att uses attention as the guided attribution technique, ER + AttR uses attention-rollout, and ER + IxG uses InputXGradient. The C prefix indicates a constrained model. Results correspond to 15 seeds. ER-C + IxG is not shown for improved clarity and can be seen in Appendix Figure \ref{['fig:sa_results_constrained']}.
• Figure 3: Yelp-50 (OOD) Kendall Rank correlations between attribution techniques for different approaches. Baseline (BS) vs Baseline serves as ground-truth for the expected correlations agreement due to seed variability.
• Figure 4: SST-Dev average AUC plausibility score per layer ($\uparrow$) with Attention and DecompX.
• Figure 5: SST-Dev$\mathcal{L}_{\text{ce}}$ vs. $\mathcal{L}_{\text{expl}}$. The vertical lines show the validation explanation-loss bounds. Each point is the average of 5 runs, the error bars show standard deviation for each loss. $\lambda$ ranges from 0 (No-ER) to 100.