SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approach

TL;DR

The paper addresses the lack of standardized XAI evaluation in NLP by extending the SIDU method to SIDU-TXT, which creates word-level heatmaps from the final CNN convolution maps using the similarity-difference $SID^c_i$ and uniqueness $U^c_i$ scores and aggregates the top-$K$ masks via $S^c_i = \frac{1}{K}\sum W^c_i \cdot M^c_i$ after thresholding with $\tau$. It implements a comprehensive three-tier evaluation—Functionally-Grounded, Human-Grounded, and Application-Grounded—across two NLP tasks: IMDB sentiment analysis and Danish asylum decision prediction, with a robust experimental setup including Hyperband optimization on an RTX-2080Ti. Results show SIDU-TXT achieves superior faithfulness in insertion/deletion tests and stronger token- and sentence-level alignment with human judgments in sentiment analysis, while providing comparable insights to Grad-CAM in the legal domain and highlighting the need for more domain-specific embeddings. The work offers a practical, domain-aware evaluation pipeline for NLP XAI and identifies key directions for improving interpretability and trust in high-stakes NLP applications.

Abstract

Explainable AI (XAI) aids in deciphering 'black-box' models. While several methods have been proposed and evaluated primarily in the image domain, the exploration of explainability in the text domain remains a growing research area. In this paper, we delve into the applicability of XAI methods for the text domain. In this context, the 'Similarity Difference and Uniqueness' (SIDU) XAI method, recognized for its superior capability in localizing entire salient regions in image-based classification is extended to textual data. The extended method, SIDU-TXT, utilizes feature activation maps from 'black-box' models to generate heatmaps at a granular, word-based level, thereby providing explanations that highlight contextually significant textual elements crucial for model predictions. Given the absence of a unified standard for assessing XAI methods, this study applies a holistic three-tiered comprehensive evaluation framework: Functionally-Grounded, Human-Grounded and Application-Grounded, to assess the effectiveness of the proposed SIDU-TXT across various experiments. We find that, in sentiment analysis task of a movie review dataset, SIDU-TXT excels in both functionally and human-grounded evaluations, demonstrating superior performance through quantitative and qualitative analyses compared to benchmarks like Grad-CAM and LIME. In the application-grounded evaluation within the sensitive and complex legal domain of asylum decision-making, SIDU-TXT and Grad-CAM demonstrate comparable performances, each with its own set of strengths and weaknesses. However, both methods fall short of entirely fulfilling the sophisticated criteria of expert expectations, highlighting the imperative need for additional research in XAI methods suitable for such domains.

Figures (5)

• Figure 1: The block diagram depicts the process of generating a feature activation text mask, serving as input for the SIDU-TXT XAI method. Within the diagram, $\langle$unk$\rangle$ symbolizes an unknown word, typically employed in NLP to indicate a missing word in a sentence. This token is mathematically assigned a value of 0.
• Figure 2: Block diagram illustrates the procedure of generating the explanations for the text data using SIDU-TXT.
• Figure 3: Three aspects of empirical assessment within the XAI Evaluation Framework for text.
• Figure 4: Annotation sample from one participant for a movie review in the dataset.
• Figure 5: Average Jaccard Similarity Scores between Human-Annotated token and XAI-Generated token score for $50$ Movie Reviews.