Few-Shot Graph Out-of-Distribution Detection with LLMs

TL;DR

This work tackles data-efficient graph OOD detection on text-attributed graphs by combining the zero-shot capabilities of large language models (LLMs) with a lightweight GNN filter. It uses LLMs to pre-filter potential out-of-distribution (OOD) nodes and to provide a small set of noisy labels, which train a GNN to infer ID status for the remainder, enabling a single-round human annotation strategy. The GNN embeddings then drive informativeness-based node selection and final ID classifier training, with optional combination of accurate and noisy labels. Across four TAG datasets, LLM-GOOD consistently outperforms baselines in ID classification and OOD detection while substantially reducing labeling costs, and analyses reveal the cost-performance trade-offs of using various LLMs and prompts for open-world annotation.

Abstract

Existing methods for graph out-of-distribution (OOD) detection typically depend on training graph neural network (GNN) classifiers using a substantial amount of labeled in-distribution (ID) data. However, acquiring high-quality labeled nodes in text-attributed graphs (TAGs) is challenging and costly due to their complex textual and structural characteristics. Large language models (LLMs), known for their powerful zero-shot capabilities in textual tasks, show promise but struggle to naturally capture the critical structural information inherent to TAGs, limiting their direct effectiveness. To address these challenges, we propose LLM-GOOD, a general framework that effectively combines the strengths of LLMs and GNNs to enhance data efficiency in graph OOD detection. Specifically, we first leverage LLMs' strong zero-shot capabilities to filter out likely OOD nodes, significantly reducing the human annotation burden. To minimize the usage and cost of the LLM, we employ it only to annotate a small subset of unlabeled nodes. We then train a lightweight GNN filter using these noisy labels, enabling efficient predictions of ID status for all other unlabeled nodes by leveraging both textual and structural information. After obtaining node embeddings from the GNN filter, we can apply informativeness-based methods to select the most valuable nodes for precise human annotation. Finally, we train the target ID classifier using these accurately annotated ID nodes. Extensive experiments on four real-world TAG datasets demonstrate that LLM-GOOD significantly reduces human annotation costs and outperforms state-of-the-art baselines in terms of both ID classification accuracy and OOD detection performance.

Figures (3)

• Figure 1: An illustration of our method. To reduce annotation costs, we use an LLM to filter out OOD nodes before selecting nodes for human annotation. The annotated ID nodes are then used to train the target ID classifier.
• Figure 2: An overview of our framework LLM-GOOD. To reduce human cost, we use LLM to filter out OOD nodes before human annotation (§ \ref{['subsec:LLM']}). To further reduce LLM cost, we use LLM to annotate a small subset of nodes, and then train a lightweight GNN filter on these noisy annotations to predict labels for the remaining nodes in the graph (§ \ref{['subsec:filter']}). After obtaining node embeddings from the GNN filter, informativeness-aware selection methods identify the most informative unlabeled potential ID nodes (§ \ref{['subsec:selection']}). After these selected nodes are annotated, the labeled accurate ID nodes are used to train the target ID classifier for ID classification and OOD detection (§ \ref{['subsec:classifier']}).
• Figure 3: ID classification and OOD detection Performance upper bound.