Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Regularization, Semi-supervision, and Supervision for a Plausible Attention-Based Explanation

Duc Hau Nguyen, Cyrielle Mallart, Guillaume Gravier, Pascale Sébillot

TL;DR

This work investigates improving the plausibility of attention-based explanations in RNN encoders by adding an attention-specific loss term to the standard classification objective. It evaluates three constraint-based approaches—entropy-based sparsity regularization, supervision from human reference annotations, and semi-supervision from heuristic maps—across ERASER datasets for NLI and text classification. The results show plausible attention can be promoted with minimal or no loss in task performance, though effects vary by dataset and annotation quality, with deeper contextualization often diminishing plausibility. The study highlights that improving explanations via attention constraints is limited by the representation space created by contextualization layers, steering future work toward more informative contextualization to support human-understandable explanations.

Abstract

Attention mechanism is contributing to the majority of recent advances in machine learning for natural language processing. Additionally, it results in an attention map that shows the proportional influence of each input in its decision. Empirical studies postulate that attention maps can be provided as an explanation for model output. However, it is still questionable to ask whether this explanation helps regular people to understand and accept the model output (the plausibility of the explanation). Recent studies show that attention weights in the RNN encoders are hardly plausible because they spread on input tokens. We thus propose 3 additional constraints to the learning objective function to improve the plausibility of the attention map: regularization to increase the attention weight sparsity, semi-supervision to supervise the map by a heuristic and supervision by human annotation. Results show that all techniques can improve the attention map plausibility at some level. We also observe that specific instructions for human annotation might have a negative effect on classification performance. Beyond the attention map, the result of experiments on text classification tasks also shows that no matter how the constraint brings the gain, the contextualization layer plays a crucial role in finding the right space for finding plausible tokens.

Regularization, Semi-supervision, and Supervision for a Plausible Attention-Based Explanation

TL;DR

This work investigates improving the plausibility of attention-based explanations in RNN encoders by adding an attention-specific loss term to the standard classification objective. It evaluates three constraint-based approaches—entropy-based sparsity regularization, supervision from human reference annotations, and semi-supervision from heuristic maps—across ERASER datasets for NLI and text classification. The results show plausible attention can be promoted with minimal or no loss in task performance, though effects vary by dataset and annotation quality, with deeper contextualization often diminishing plausibility. The study highlights that improving explanations via attention constraints is limited by the representation space created by contextualization layers, steering future work toward more informative contextualization to support human-understandable explanations.

Abstract

Attention mechanism is contributing to the majority of recent advances in machine learning for natural language processing. Additionally, it results in an attention map that shows the proportional influence of each input in its decision. Empirical studies postulate that attention maps can be provided as an explanation for model output. However, it is still questionable to ask whether this explanation helps regular people to understand and accept the model output (the plausibility of the explanation). Recent studies show that attention weights in the RNN encoders are hardly plausible because they spread on input tokens. We thus propose 3 additional constraints to the learning objective function to improve the plausibility of the attention map: regularization to increase the attention weight sparsity, semi-supervision to supervise the map by a heuristic and supervision by human annotation. Results show that all techniques can improve the attention map plausibility at some level. We also observe that specific instructions for human annotation might have a negative effect on classification performance. Beyond the attention map, the result of experiments on text classification tasks also shows that no matter how the constraint brings the gain, the contextualization layer plays a crucial role in finding the right space for finding plausible tokens.
Paper Structure (11 sections, 5 equations, 5 figures)

This paper contains 11 sections, 5 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Tasks and Datasets
  4. Attention Mechanism on RNN Encoders
  5. Constraints on the Objective Function
  6. Sparsity Regularization
  7. Supervision from Reference Annotation
  8. Semi-supervision from Heuristics
  9. Implementation and Training Parameters
  10. Experimental Results
  11. Conclusion

Figures (5)

  • Figure 1: Generic architecture of a RNN-based attention model for classification.
  • Figure 2: Plausibility (AUPRC, top) and performance (F-score, bottom) on HateXPlain, YelpHat-50, and e-SNLI.
  • Figure 3: Examples of attention maps on one of the e-SNLI entailment pair.
  • Figure 4: Recall (top) and specificity (bottom) of attention map against annotation.
  • Figure 5: Plausibility and performance in 3 datasets, for 3 techniques, for 3 settings.