Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications

TL;DR

This survey addresses the challenge of combining large language models with text-attributed graphs by framing two core primitives: LLM for TAG and TAG for LLM. It offers a comprehensive taxonomy of orchestration frameworks, including sequential, parallel, and multi-module designs, and discusses TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. The authors synthesize empirical observations across diverse applications, curate available datasets, and map real-world use cases in domains such as biomedicine, chemistry, and knowledge-intensive question answering. They further outline open challenges—explicit topology handling, robustness, and scalability—and propose future directions toward TAG foundation models, autonomous agents, advanced data management, and retrieval-grounded reasoning. Overall, the work provides a practical, theory-grounded blueprint for advancing structure-aware language intelligence at the intersection of language and graph learning.

Abstract

Large Language Models (LLMs) have achieved remarkable success in natural language processing through strong semantic understanding and generation. However, their black-box nature limits structured and multi-hop reasoning. In contrast, Text-Attributed Graphs (TAGs) provide explicit relational structures enriched with textual context, yet often lack semantic depth. Recent research shows that combining LLMs and TAGs yields complementary benefits: enhancing TAG representation learning and improving the reasoning and interpretability of LLMs. This survey provides the first systematic review of LLM--TAG integration from an orchestration perspective. We introduce a novel taxonomy covering two fundamental directions: LLM for TAG, where LLMs enrich graph-based tasks, and TAG for LLM, where structured graphs improve LLM reasoning. We categorize orchestration strategies into sequential, parallel, and multi-module frameworks, and discuss advances in TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. Beyond methodology, we summarize empirical insights, curate available datasets, and highlight diverse applications across recommendation systems, biomedical analysis, and knowledge-intensive question answering. Finally, we outline open challenges and promising research directions, aiming to guide future work at the intersection of language and graph learning.

Figures (7)

• Figure 1: Synergistic improvement with the orchestration of techniques for large language models and text-attributed graphs.
• Figure 2: Holistic categorisation of approaches for Large Language Models and Text-Attributed Graphs.
• Figure 3: The illustration of sequential orchestration of LLM4TAG approaches, which contain (a) Structure-agnostic prompts, (b) Structure-aware prompts, (c) Structure-aware language model (LM), and (d) Mixture of experts.
• Figure 4: The illustration of parallel orchestration of LLM4TAG approaches, which contain (a) Two-tower structure, and (b) Mutual supervision via pseudo-labels.
• Figure 5: The illustration of pre-trained models for TAGs of LLM4TAG approaches, which contain (a) TAG Transformers, (b) Graph foundation models, (c) Fine-tuning, (d) Graph prompt tuning.