Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Combating Spurious Correlations in Graph Interpretability via Self-Reflection

Kecheng Cai, Chenyang Xu, Chao Peng

TL;DR

This work tackles explanation-level reliability in interpretable graph learning under spurious correlations by introducing a training-free self-reflection framework that iteratively refines edge importance via a multiplicatively updated mask $Z^{(t)}$ fed back into the upstream module, aligning with a mutual-information objective framework that balances fidelity and sparsity. A key theoretical result shows there exists a consistent, optimal mask sequence across iterations, motivating a consistency-based fine-tuning objective ${\mathcal L}_{con}$ that improves downstream performance, notably the AUC, across challenging datasets like Spurious-Motif and MolHIV. Empirically, self-reflection enhances accuracy particularly under strong spurious bias, with AUC remaining stable or increasing when combined with fine-tuning; however, gains depend on dataset complexity and the masking dynamics. Overall, the approach provides a practical, architecture-agnostic path toward more faithful explanations in graph reasoning, with potential for RL-inspired exploration in future work.

Abstract

Interpretable graph learning has recently emerged as a popular research topic in machine learning. The goal is to identify the important nodes and edges of an input graph that are crucial for performing a specific graph reasoning task. A number of studies have been conducted in this area, and various benchmark datasets have been proposed to facilitate evaluation. Among them, one of the most challenging is the Spurious-Motif benchmark, introduced at ICLR 2022. The datasets in this synthetic benchmark are deliberately designed to include spurious correlations, making it particularly difficult for models to distinguish truly relevant structures from misleading patterns. As a result, existing methods exhibit significantly worse performance on this benchmark compared to others. In this paper, we focus on improving interpretability on the challenging Spurious-Motif datasets. We demonstrate that the self-reflection technique, commonly used in large language models to tackle complex tasks, can also be effectively adapted to enhance interpretability in datasets with strong spurious correlations. Specifically, we propose a self-reflection framework that can be integrated with existing interpretable graph learning methods. When such a method produces importance scores for each node and edge, our framework feeds these predictions back into the original method to perform a second round of evaluation. This iterative process mirrors how large language models employ self-reflective prompting to reassess their previous outputs. We further analyze the reasons behind this improvement from the perspective of graph representation learning, which motivates us to propose a fine-tuning training method based on this feedback mechanism.

Combating Spurious Correlations in Graph Interpretability via Self-Reflection

TL;DR

This work tackles explanation-level reliability in interpretable graph learning under spurious correlations by introducing a training-free self-reflection framework that iteratively refines edge importance via a multiplicatively updated mask fed back into the upstream module, aligning with a mutual-information objective framework that balances fidelity and sparsity. A key theoretical result shows there exists a consistent, optimal mask sequence across iterations, motivating a consistency-based fine-tuning objective that improves downstream performance, notably the AUC, across challenging datasets like Spurious-Motif and MolHIV. Empirically, self-reflection enhances accuracy particularly under strong spurious bias, with AUC remaining stable or increasing when combined with fine-tuning; however, gains depend on dataset complexity and the masking dynamics. Overall, the approach provides a practical, architecture-agnostic path toward more faithful explanations in graph reasoning, with potential for RL-inspired exploration in future work.

Abstract

Interpretable graph learning has recently emerged as a popular research topic in machine learning. The goal is to identify the important nodes and edges of an input graph that are crucial for performing a specific graph reasoning task. A number of studies have been conducted in this area, and various benchmark datasets have been proposed to facilitate evaluation. Among them, one of the most challenging is the Spurious-Motif benchmark, introduced at ICLR 2022. The datasets in this synthetic benchmark are deliberately designed to include spurious correlations, making it particularly difficult for models to distinguish truly relevant structures from misleading patterns. As a result, existing methods exhibit significantly worse performance on this benchmark compared to others. In this paper, we focus on improving interpretability on the challenging Spurious-Motif datasets. We demonstrate that the self-reflection technique, commonly used in large language models to tackle complex tasks, can also be effectively adapted to enhance interpretability in datasets with strong spurious correlations. Specifically, we propose a self-reflection framework that can be integrated with existing interpretable graph learning methods. When such a method produces importance scores for each node and edge, our framework feeds these predictions back into the original method to perform a second round of evaluation. This iterative process mirrors how large language models employ self-reflective prompting to reassess their previous outputs. We further analyze the reasons behind this improvement from the perspective of graph representation learning, which motivates us to propose a fine-tuning training method based on this feedback mechanism.
Paper Structure (35 sections, 1 theorem, 17 equations, 4 figures, 5 tables)

This paper contains 35 sections, 1 theorem, 17 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Other Related Works
  4. Graph Neural Networks.
  5. Post-hoc Explanation.
  6. Beyond Self-Reflection: Mitigating LLM Hallucinations.
  7. Preliminaries
  8. Problem Definition
  9. L2X Architecture
  10. Spurious-Motif Benchmark
  11. Self-Reflection for Interpretable Graph Learning
  12. Monotonicity Remark.
  13. Empirical Study of Self-Reflection
  14. Setup
  15. Datasets.
  16. ...and 20 more sections

Key Result

Theorem 1

There always exists a set of optimal masks $\{Z^{(t)}\}_{t \in [k]}$ to Problem eq:rif that maintains consistency, i.e., $Z^{(1)} = Z^{(2)} = \cdots = Z^{(k)}$.

Figures (4)

  • Figure 1: An illustration of the self-reflection framework.
  • Figure 2: Performance trends under the self-reflection framework. From left to right, the plots correspond to datasets with spurious correlation levels $b = 0.5$, $0.7$, and $0.9$, respectively.
  • Figure 3: An illustration of how edge importance scores evolve across self-reflection iterations. The top row shows the average edge scores for positive samples, while the bottom row shows those for negative samples. From left to right, the plots correspond to datasets with spurious correlation levels $b = 0.5$, $0.7$, and $0.9$, respectively.
  • Figure 4: Performance trends under the self-reflection framework without multiplication. From left to right, the plots correspond to datasets with spurious correlation levels $b = 0.5$, $0.7$, and $0.9$, respectively.

Theorems & Definitions (2)

  • Theorem 1
  • proof