Toward Efficient Testing of Graph Neural Networks via Test Input Prioritization

TL;DR

GraphRank tackles the reliability of Graph Neural Networks by prioritizing test inputs with a novel, model-agnostic plus graph-structure-informed framework. It combines model-aware attributes from the target GNN with model-agnostic attributes derived from graph data, then enhances them via neighbor aggregation and a learnable binary classifier trained iteratively under a fixed labeling budget. The approach is validated across large-scale, small-scale, and heterophilic graphs, with comprehensive ablations and efficiency analyses showing superior performance to baselines and practical time costs. Overall, GraphRank offers a scalable, effective strategy for uncovering GNN failures and guiding targeted model repair under budget constraints.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable efficacy in handling graph-structured data; however, they exhibit failures after deployment, which can cause severe consequences. Hence, conducting thorough testing before deployment becomes imperative to ensure the reliability of GNNs. However, thorough testing requires numerous manually annotated test data. To mitigate the annotation cost, strategically prioritizing and labeling high-quality unlabeled inputs for testing becomes crucial, which facilitates uncovering more model failures with a limited labeling budget. Unfortunately, existing test input prioritization techniques either overlook the valuable information contained in graph structures or are overly reliant on attributes extracted from the target model, i.e., model-aware attributes, whose quality can vary significantly. To address these issues, we propose a novel test input prioritization framework, named GraphRank, for GNNs. GraphRank introduces model-agnostic attributes to compensate for the limitations of the model-aware ones. It also leverages the graph structure information to aggregate attributes from neighboring nodes, thereby enhancing the model-aware and model-agnostic attributes. Furthermore, GraphRank combines the above attributes with a binary classifier, using it as a ranking model to prioritize inputs. This classifier undergoes iterative training, which enables it to learn from each round's feedback and improve its performance accordingly. Extensive experiments demonstrate GraphRank's superiority over existing techniques.

Figures (6)

• Figure 1: A motivational example. We set the budget for selecting nodes to be three. The shaded nodes are selected nodes, while the remaining nodes are unselected. The square nodes are failures, and the circles are correctly classified
• Figure 2: The overview of GraphRank. 1) Attribute exploration: explore both model-aware and model-agnostic attributes; 2) Attributes enhancement: enhance attributes with graph structure; 3) Nodes ranking: train a classifier for priority score estimation and ranking
• Figure 3: The distribution of neighbor failure rates for both correctness nodes set and failure nodes set. X-axis: the neighbor failure rates. Y-axis: the proportion of nodes corresponding to different neighbor failure rates
• Figure 4: The TRC curves of GraphRank and the baseline methods. X-axis: The ratio of budget allocation to the total number of failures. Y-axis: TRC value (%)
• Figure 5: The distribution of each priority metric on failure nodes and correctness nodes. X-axis: the corresponding values of each priority metric. Y-axis: the proportion of nodes (%)