Influence Functions for Edge Edits in Non-Convex Graph Neural Networks

TL;DR

This work introduces a proximal Bregman-based influence function tailored for edge edits in non-convex graph neural networks, enabling accurate prediction of how single or multiple edge deletions/insertions affect predictions and evaluation metrics. The method decomposes influence into parameter-shift and message-propagation components and uses a generalized Gauss–Newton matrix with LiSSA for scalable inversion, addressing non-convexity and graph-structure changes. Empirical results on multiple real-world datasets demonstrate high concordance between predicted and actual influence (up to ~0.95) and show applicability to adversarial attacks, node-embedding analysis, and graph rewiring. The approach offers a unified, propagation-aware view of edge importance that can guide edge editing for improved validation loss and reveal the roles of homophilic versus heterophilic connections, though scalability to many simultaneous edits and very deep GNNs remains a challenge.

Abstract

Understanding how individual edges influence the behavior of graph neural networks (GNNs) is essential for improving their interpretability and robustness. Graph influence functions have emerged as promising tools to efficiently estimate the effects of edge deletions without retraining. However, existing influence prediction methods rely on strict convexity assumptions, exclusively consider the influence of edge deletions while disregarding edge insertions, and fail to capture changes in message propagation caused by these modifications. In this work, we propose a proximal Bregman response function specifically tailored for GNNs, relaxing the convexity requirement and enabling accurate influence prediction for standard neural network architectures. Furthermore, our method explicitly accounts for message propagation effects and extends influence prediction to both edge deletions and insertions in a principled way. Experiments with real-world datasets demonstrate accurate influence predictions for different characteristics of GNNs. We further demonstrate that the influence function is versatile in applications such as graph rewiring and adversarial attacks.

Figures (10)

• Figure 1: An illustration of beneficial and harmful edges identified by an influence function proposed in this work with respect to two evaluation metrics. A harmful edge is one that either blocks information propagation between nodes (over-squashing alon2020bottleneck) or makes node representations indistinguishable (over-smoothing li2018deeper). The barbell graph consists of two clusters, each with distinct node labels. The influence function can be used to analyze the properties of edges. For example, the edges connecting two different clusters mitigate over-squashing while amplifying over-smoothing.
• Figure 2: Predicted influence versus actual influence on a four-layer GCN. The x-axis represents the predicted influence, the y-axis represents the actual influence, and the dotted line represents the perfect alignment.
• Figure 3: The relationship between parameter shift effect and message propagation effect defined in \ref{['eqn:ours_grad']}. The x-axis denotes the parameter shift effect, and the y-axis denotes the message propagation effect.
• Figure 4: Predicted influence versus actual influence on a four-layer GCN under varying numbers of inserted edges.
• Figure 5: Mean influence of homophilic and heterophilic edges on validation loss for edge insertion (left) and edge deletion (right). Each dumbbell connects the average influence of homophilic (light green) and heterophilic (dark green) edges across six datasets. A negative value indicates that the edge edit decreases the validation loss, thus improving the performance.

Theorems & Definitions (1)

• Remark