Semantic Refinement with LLMs for Graph Representations

TL;DR

This work addresses the challenge of structure-semantics heterogeneity in graphs by shifting adaptation from the model to the data: node semantics are treated as task-adaptive variables refined in a closed loop with a fixed GNN and a large language model. The Data-Adaptive Semantic Refinement (DAS) framework uses a memory-augmented process to retrieve in-graph exemplars and guide LLM-based semantic refinement, which is then fed back to update the GNN input representations. Across text-attributed and text-free graphs, DAS yields consistent gains on structure-dominated domains while remaining competitive on semantics-rich graphs, and demonstrates improved transfer performance in domain adaptation settings. Theoretical grounding via a Majorization-Minimization interpretation explains why the iterative refinement improves alignment with the target structure-semantics regime, while mechanism analyses highlight the importance of joint semantic–structural exemplar retrieval and memory. Overall, the paper proposes a novel data-centric paradigm for graph learning, emphasizing adaptive input representations rather than fixed architectural biases.

Abstract

Graph-structured data exhibit substantial heterogeneity in where their predictive signals originate: in some domains, node-level semantics dominate, while in others, structural patterns play a central role. This structure-semantics heterogeneity implies that no graph learning model with a fixed inductive bias can generalize optimally across diverse graph domains. However, most existing methods address this challenge from the model side by incrementally injecting new inductive biases, which remains fundamentally limited given the open-ended diversity of real-world graphs. In this work, we take a data-centric perspective and treat node semantics as a task-adaptive variable. We propose a Data-Adaptive Semantic Refinement framework DAS for graph representation learning, which couples a fixed graph neural network (GNN) and a large language model (LLM) in a closed feedback loop. The GNN provides implicit supervisory signals to guide the semantic refinement of LLM, and the refined semantics are fed back to update the same graph learner. We evaluate our approach on both text-rich and text-free graphs. Results show consistent improvements on structure-dominated graphs while remaining competitive on semantics-rich graphs, demonstrating the effectiveness of data-centric semantic adaptation under structure-semantics heterogeneity.

Figures (6)

• Figure 1: Structure--semantics heterogeneity and data-centric adaptation. (a) Real-world graphs vary widely in their reliance on semantic and structural patterns as sources of predictive signal. (b) Model-centric approaches with fixed inductive biases become misaligned when deployed across graphs with different structure--semantics regimes, leading to poor generalization. (c) In contrast, our proposed DAS keeps the graph model fixed and iteratively refines node semantics through prediction-driven feedback, adapting the data representation to the target task.
• Figure 2: Overview of the DAS framework. Node descriptions are iteratively refined through a closed loop between a fixed GNN and an LLM. At each iteration, the GNN provides task feedback and a model-conditioned memory retrieves structurally and semantically aligned in-graph exemplars, which guide the LLM to update node semantics before feeding them back to the same GNN.
• Figure 3: The impact of the number of iterations.
• Figure 4: Hyper-parameter sensitivity on $\alpha$.
• Figure 5: Fixed verbalization template used to construct the topological summary in node initialization.

Theorems & Definitions (4)

• Lemma B.1: Majorization and Tightness
• proof
• Theorem B.2: Monotonic Descent
• proof