MASE: Interpretable NLP Models via Model-Agnostic Saliency Estimation

TL;DR

This work tackles the challenge of interpreting NLP deep models by introducing Model-agnostic Saliency Estimation (MASE), which performs perturbations in the embedding space via Normalized Linear Gaussian Perturbations (NLGP) to obtain local, model-agnostic explanations. MASE yields a saliency vector by fitting a local linear model around the input embeddings and supports a sparse variant (MASE_Sparse) for faster, focused interpretations. The framework unifies several existing explanation methods (e.g., LIME, SHAP, Integrated Gradients) under a common perturbation-and-approximation paradigm and demonstrates superior delta-accuracy faithfulness on LSTM and BERT models across IMDB and Reuters. The approach provides a practical, theoretically-grounded tool for transparent NLP decision-making with broad applicability to text classification tasks.

Abstract

Deep neural networks (DNNs) have made significant strides in Natural Language Processing (NLP), yet their interpretability remains elusive, particularly when evaluating their intricate decision-making processes. Traditional methods often rely on post-hoc interpretations, such as saliency maps or feature visualization, which might not be directly applicable to the discrete nature of word data in NLP. Addressing this, we introduce the Model-agnostic Saliency Estimation (MASE) framework. MASE offers local explanations for text-based predictive models without necessitating in-depth knowledge of a model's internal architecture. By leveraging Normalized Linear Gaussian Perturbations (NLGP) on the embedding layer instead of raw word inputs, MASE efficiently estimates input saliency. Our results indicate MASE's superiority over other model-agnostic interpretation methods, especially in terms of Delta Accuracy, positioning it as a promising tool for elucidating the operations of text-based models in NLP.

Figures (3)

• Figure 1: Pipelines of MASE vs. other perturbation-based methods. Left: Pipeline of other perturbation-based methods. Perturbations are often made at the word level and use a special token to represent the absence of words. Right: Pipeline of MASE. We perturb the embedding of words instead of the raw words so that the perturbation space is largely expanded from binary to Euclidean, which is more precise to describe the local behavior of the target model.
• Figure 2: Illustration of Normalized Linear Gaussian Perturbation Procedure in 3-dimensional Cartesian Space. (a) shows the original words of the toy example and their corresponding word embedding vectors in 3-D. (b) illustrates the normalized embedding vectors. (c) displays the embedding vectors after applying the Gaussian perturbation.
• Figure 3: Experimental results of investigating the effectiveness of embedding-based estimation, different masking sizes, and perturbation sizes. Column 1, column 2, and column 3 are conducted on the IMDB dataset with the LSTM model, the Reuters dataset with the LSTM model, and IMDB dataset with the BERT-based model, respectively.

Theorems & Definitions (10)

• Definition 3.1: Model-Agnostic Saliency Estimation (MASE)
• Definition 3.2: Normalized Linear Gaussian Perturbation
• Theorem 3.1
• Definition 3.3: Infidelity
• Theorem 3.2
• Definition 3.4: Sparse Model-Agnostic Saliency Estimation (MASE_Sparse)
• Proposition 3.1
• Proposition 3.2
• Proposition 3.3
• Proposition 3.4