NoMatterXAI: Generating "No Matter What" Alterfactual Examples for Explaining Black-Box Text Classification Models

TL;DR

NoMatterXAI tackles the gap in explainable AI for text by automatically generating alterfactual explanations (AEs) that change irrelevant features while preserving model output. It formulates AE generation as an optimization problem and solves it with a greedy two-step algorithm: identify irrelevant words and replace them with contextually opposite words via ConceptNet or large language models, while enforcing semantic-field and grammar constraints. The approach achieves high fidelity (up to 95%) and strong context preservation (over 90%), with a quantified trade-off between perturbation breadth and runtime. A human user study shows AEs can help end users compare model biases, supporting practical deployment decisions and bias auditing. Overall, NoMatterXAI provides a principled, scalable method for revealing irrelevancies in text classifier decisions and enabling bias-aware model comparison.

Abstract

In Explainable AI (XAI), counterfactual explanations (CEs) are a well-studied method to communicate feature relevance through contrastive reasoning of "what if" to explain AI models' predictions. However, they only focus on important (i.e., relevant) features and largely disregard less important (i.e., irrelevant) ones. Such irrelevant features can be crucial in many applications, especially when users need to ensure that an AI model's decisions are not affected or biased against specific attributes such as gender, race, religion, or political affiliation. To address this gap, the concept of alterfactual explanations (AEs) has been proposed. AEs explore an alternative reality of "no matter what", where irrelevant features are substituted with alternative features (e.g., "republicans" -> "democrats") within the same attribute (e.g., "politics") while maintaining a similar prediction output. This serves to validate whether AI model predictions are influenced by the specified attributes. Despite the promise of AEs, there is a lack of computational approaches to systematically generate them, particularly in the text domain, where creating AEs for AI text classifiers presents unique challenges. This paper addresses this challenge by formulating AE generation as an optimization problem and introducing MoMatterXAI, a novel algorithm that generates AEs for text classification tasks. Our approach achieves high fidelity of up to 95% while preserving context similarity of over 90% across multiple models and datasets. A human study further validates the effectiveness of AEs in explaining AI text classifiers to end users. All codes will be publicly available.

Figures (7)

• Figure 1: A comparison of various AI explanation algorithms, including Counterfactual, Semifactual, and our proposed Alterfactals explanations. Alterfactual explanations aim to validate whether AI model predictions are influenced by specific attributes such as race or gender.
• Figure 2: AE generation of $x^*$ (orange circle) from $x$ (grey circle) by perturbing irrelevant features $x_\mathrm{ir}$ of $x$ within their semantic fields while still maintaining original context of $x$.
• Figure 3: Trade-off between $L_2$ distance between word embeddings of original and perturbed token versus Flip Rate--i.e., the chance of perturbed token converting to new word, and context similarity (SIM) on DistilBERT with JIG dataset.
• Figure 4: NoMatterXAI's fidelity has a strong negative correlation (correlation coefficient corr${\le}$-0.7) with the empirical gender bias evaluation (scores are normalized to [0,1])
• Figure 5: Trade-off between runtime, number of queries, fidelity and context similarity per input, and number of model queries averaged across all datasets and target models.

Theorems & Definitions (1)

• Definition 1