A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference

TL;DR

The paper investigates whether attention maps provide plausible explanations for decisions in natural language inference (NLI). It compares cross-attention between two LSTM encoders against human-highlight annotations from eSNLI and against a cosine-similarity heuristic designed to identify semantically related words across sentences. Findings show that the heuristic aligns better with human annotations than the model-based attention, which exhibits low plausibility due to attention spreading on less informative tokens; this highlights the need for embedding-driven cues to improve interpretability. The work suggests using the heuristic as an evaluation tool and potentially as a regularizer to enhance plausibility without compromising model faithfulness, informing future explainability approaches in sentence-pair tasks.

Abstract

Attention maps in neural models for NLP are appealing to explain the decision made by a model, hopefully emphasizing words that justify the decision. While many empirical studies hint that attention maps can provide such justification from the analysis of sound examples, only a few assess the plausibility of explanations based on attention maps, i.e., the usefulness of attention maps for humans to understand the decision. These studies furthermore focus on text classification. In this paper, we report on a preliminary assessment of attention maps in a sentence comparison task, namely natural language inference. We compare the cross-attention weights between two RNN encoders with human-based and heuristic-based annotations on the eSNLI corpus. We show that the heuristic reasonably correlates with human annotations and can thus facilitate evaluation of plausible explanations in sentence comparison tasks. Raw attention weights however remain only loosely related to a plausible explanation.

Figures (7)

• Figure 1: Example of a heuristic-based plausible attention map.
• Figure 2: Overall model architecture. The dash lines indicate that layers share the weights; in other words, we use the same first 3 layers ($Embedding$, $Contextualization$, $Attention$) for both $premise$ and $hypothesis$.
• Figure 3: The attention mechanism used in the experimentation. The intuition is that $\alpha$ vector should give how much relevant a word is comparing to the sentence embedding of the other side $\overline{h_m}$.
• Figure 4: ROC curve that measures how much a model matches the ground truth. The orange (resp. green) line compares the heuristic (resp. model-based )map with the human annotation. The dash blue line in diagonal is the worst prediction, where a system highlights every words.
• Figure 5: AUC of human annotation map (orange) and model attention (blue) as a function of $\epsilon_{\hbox{\tiny heuristic}}$.