Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models

TL;DR

This work tackles the challenge of evaluating XAI explanations for encoder-based language models by introducing a general framework with four metrics: Human-reasoning Agreement, Robustness, Consistency, and Contrastivity. It systematically compares six XAI techniques spanning five categories across five encoder-based LMs and two sentiment datasets (IMDB and TSE), using a set of ranking- and distribution-based measures (MAP, AP, AD, MAD, Spearman's rho, and KL divergence). Key findings show LIME excels in aligning explanations with human rationale and maintaining consistency, AMV delivers exceptional robustness and stability, and LRP achieves strong contrastivity for complex models; no single method dominates all metrics. The framework and findings offer a standardized benchmark to guide technique selection and enhance transparency in NLP systems employing encoder-based architectures.

Abstract

The black-box nature of large language models (LLMs) necessitates the development of eXplainable AI (XAI) techniques for transparency and trustworthiness. However, evaluating these techniques remains a challenge. This study presents a general evaluation framework using four key metrics: Human-reasoning Agreement (HA), Robustness, Consistency, and Contrastivity. We assess the effectiveness of six explainability techniques from five different XAI categories model simplification (LIME), perturbation-based methods (SHAP), gradient-based approaches (InputXGradient, Grad-CAM), Layer-wise Relevance Propagation (LRP), and attention mechanisms-based explainability methods (Attention Mechanism Visualization, AMV) across five encoder-based language models: TinyBERT, BERTbase, BERTlarge, XLM-R large, and DeBERTa-xlarge, using the IMDB Movie Reviews and Tweet Sentiment Extraction (TSE) datasets. Our findings show that the model simplification-based XAI method (LIME) consistently outperforms across multiple metrics and models, significantly excelling in HA with a score of 0.9685 on DeBERTa-xlarge, robustness, and consistency as the complexity of large language models increases. AMV demonstrates the best Robustness, with scores as low as 0.0020. It also excels in Consistency, achieving near-perfect scores of 0.9999 across all models. Regarding Contrastivity, LRP performs the best, particularly on more complex models, with scores up to 0.9371.

Figures (10)

• Figure 1: An overview of our comprehensive XAI evaluation framework for assessing the effectiveness of explainability techniques across different scenarios.
• Figure 2: The selected transformer-based models by complexity band parameter size.
• Figure 3: SHAP explanation of BERT_base model's IMDB movie sentiment prediction. Positive words are highlighted in red and negative words in blue.
• Figure 4: LIME explanation of BERT_large model's IMDB movie sentiment prediction. Positive words are highlighted in orange and negative words in blue.
• Figure 5: Grad-CAM explanation of DeBERTa model's positive sentiment prediction on IMDB movie review (Yellow indicates high importance and dark purple indicates less importance).