Natural Language Counterfactual Explanations for Graphs Using Large Language Models

TL;DR

This paper tackles making graph counterfactual explanations accessible via natural language using open-source LLMs. It formulates the problem, introduces an LLM-based translation pipeline, and proposes five evaluation metrics (TNI, CCI, FTNF, CFTNF, CFTNN) plus human judgments. Empirical results show larger LLMs (up to 14B) deliver more accurate and coherent explanations across CF-GNNExplainer and CF-GNNFeatures on Cora and CiteSeer. The framework is explainer-agnostic and promotes transparency in graph-based ML, with potential for domain-specific fine-tuning in future work.

Abstract

Explainable Artificial Intelligence (XAI) has emerged as a critical area of research to unravel the opaque inner logic of (deep) machine learning models. Among the various XAI techniques proposed in the literature, counterfactual explanations stand out as one of the most promising approaches. However, these "what-if" explanations are frequently complex and technical, making them difficult for non-experts to understand and, more broadly, challenging for humans to interpret. To bridge this gap, in this work, we exploit the power of open-source Large Language Models to generate natural language explanations when prompted with valid counterfactual instances produced by state-of-the-art explainers for graph-based models. Experiments across several graph datasets and counterfactual explainers show that our approach effectively produces accurate natural language representations of counterfactual instances, as demonstrated by key performance metrics.

Figures (2)

• Figure 1: On the left, the factual graph. On the right, the counterfactual graph computed using CF-GNNExplainer. Each graph contains nodes ids and classes
• Figure 2: Prompt example to get the explanations