Robustness in Text-Attributed Graph Learning: Insights, Trade-offs, and New Defenses

TL;DR

This work addresses robustness in text-attributed graph learning by conducting a unified, large-scale evaluation of GNNs, RGNNs, and GraphLLMs across ten datasets in four domains under both poisoning and evasion attacks. It reveals a persistent text-structure trade-off, demonstrates how text encoders influence defense effectiveness, and shows GraphLLMs are particularly vulnerable to training-data corruption. To mitigate these issues, the authors introduce SFT-auto, a novel LLM-based framework that detects attacks and provides recovery within a single model, achieving balanced robustness across textual and structural perturbations. The study provides a practical foundation for TAG security and offers actionable guidance for robust TAG learning in adversarial settings.

Abstract

While Graph Neural Networks (GNNs) and Large Language Models (LLMs) are powerful approaches for learning on Text-Attributed Graphs (TAGs), a comprehensive understanding of their robustness remains elusive. Current evaluations are fragmented, failing to systematically investigate the distinct effects of textual and structural perturbations across diverse models and attack scenarios. To address these limitations, we introduce a unified and comprehensive framework to evaluate robustness in TAG learning. Our framework evaluates classical GNNs, robust GNNs (RGNNs), and GraphLLMs across ten datasets from four domains, under diverse text-based, structure-based, and hybrid perturbations in both poisoning and evasion scenarios. Our extensive analysis reveals multiple findings, among which three are particularly noteworthy: 1) models have inherent robustness trade-offs between text and structure, 2) the performance of GNNs and RGNNs depends heavily on the text encoder and attack type, and 3) GraphLLMs are particularly vulnerable to training data corruption. To overcome the identified trade-offs, we introduce SFT-auto, a novel framework that delivers superior and balanced robustness against both textual and structural attacks within a single model. Our work establishes a foundation for future research on TAG security and offers practical solutions for robust TAG learning in adversarial environments. Our code is available at: https://github.com/Leirunlin/TGRB.

Figures (11)

• Figure 1: The overall framework for evaluating the robustness of TAG learning.
• Figure 2: Comparison of robustness against structural attacks. Ranks are averaged across datasets (excluding failures) based on: 1) absolute accuracy under attack (lower rank is better), and 2) relative accuracy drop from the clean baseline (lower drop rank is better).
• Figure 3: Comparison of robustness against textual attacks. Ranks are averaged across datasets (excluding failures) based on: 1) absolute accuracy under attack (lower rank is better), and 2) relative accuracy drop from the clean baseline (lower drop rank is better).
• Figure 4: The text-structure robustness trade-off. (Left) SFT-auto uniquely balances this trade-off, while baseline models are polarized towards either text or structure robustness. (Right) On text-friendly datasets (e.g., PubMed), models are less reliant on structure and thus less vulnerable to its perturbation. The opposite holds for structure-critical datasets (e.g., Computer, Photo).
• Figure 5: Performance of robust variants against base GraphLLMs. Points represent the improvement compared to base models against structural (x-axis) and textual (y-axis) evasion attacks.