GraphXAIN: Narratives to Explain Graph Neural Networks

TL;DR

GraphXAIN tackles the interpretability gap in graph neural networks by transforming traditional explanations (subgraphs and feature importances) into coherent natural-language narratives using large language models. The method is model- and explainer-agnostic, complementing existing graph explainers such as GNNExplainer, and is demonstrated on node classification (NBA dataset) and node regression (IMDB dataset) tasks. A human study shows that narratives improve understandability, satisfaction, convincingness, and communicability, with the combined use of GraphXAIN and GNNExplainer yielding the broadest enhancements across eight explainability dimensions. The work highlights the potential of narrative-based explanations to make graph-based AI more accessible to practitioners and non-expert users, while outlining future directions such as counterfactual narratives and addressing potential language-model hallucinations.

Abstract

Graph Neural Networks (GNNs) are a powerful technique for machine learning on graph-structured data, yet they pose challenges in interpretability. Existing GNN explanation methods usually yield technical outputs, such as subgraphs and feature importance scores, that are difficult for non-data scientists to understand and thereby violate the purpose of explanations. Motivated by recent Explainable AI (XAI) research, we propose GraphXAIN, a method that generates natural language narratives explaining GNN predictions. GraphXAIN is a model- and explainer-agnostic method that uses Large Language Models (LLMs) to translate explanatory subgraphs and feature importance scores into coherent, story-like explanations of GNN decision-making processes. Evaluations on real-world datasets demonstrate GraphXAIN's ability to improve graph explanations. A survey of machine learning researchers and practitioners reveals that GraphXAIN enhances four explainability dimensions: understandability, satisfaction, convincingness, and suitability for communicating model predictions. When combined with another graph explainer method, GraphXAIN further improves trustworthiness, insightfulness, confidence, and usability. Notably, 95% of participants found GraphXAIN to be a valuable addition to the GNN explanation method. By incorporating natural language narratives, our approach serves both graph practitioners and non-expert users by providing clearer and more effective explanations.

Figures (9)

• Figure 1: LLM-generated GraphXAIN (bottom) complements GNNExplainer'sgnnexplainergeneratingexplanationsgraph subgraph and feature importance outputs (top) in explaining the GNN's prediction for the player 57 (node 57) in the NBA dataset, based on the player's attributes, field statistics, and social connections with other players (nodes).
• Figure 2: Workflow diagram of the GraphXAIN method. Graph-structured data, along with its corresponding features, is first processed by a GNN model. Next, a graph explainer generates an explanatory subgraph and corresponding feature importance values for a target node. The dataset description, target node features and edge connections, the GNN prediction score, the explanatory subgraph, and the feature importance values are then incorporated into a prompt. This prompt is processed by the LLM, which generates GraphXAIN, a complementary narrative to explain GNN's prediction.
• Figure 3: LLM-generated GraphXAIN (bottom) complements GNNExplainer's gnnexplainergeneratingexplanationsgraph subgraph and feature importance outputs (top) in explaining the GNN’s prediction for the movie "$8\frac{1}{2}$" (node 432) in the IMDB dataset, based on movie features and connections with other movies (nodes) via shared actors.
• Figure 4: LLM-generated XAI Description (bottom) for the GNNExplainer's gnnexplainergeneratingexplanationsgraph subgraph and feature importance outputs (top) to explain the GNN's prediction for player (node) 57. Compared to GraphXAIN's output, XAI Description focuses on features and connections in a static and context-free format, which is less valuable than a XAI Narrative.
• Figure 5: The prompt schema used to generate GraphXAINs.

Theorems & Definitions (3)

• Definition 1: XAI Narrative
• Definition 2: XAI Description
• Definition 3: GraphXAIN