Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression

TL;DR

This work introduces HS2C, a homophily-aware compression framework for text-attributed graphs (TAGs) to enhance large language model (LLM) reasoning under strict input contexts. It combines an entropy-guided, hierarchical structure partition to uncover cohesive, homophilic communities with a semantic-aggregation step that distills background text per community, producing a compact yet informative graph for downstream tasks. Empirical results across ten node-level benchmarks show substantial improvements in compression efficiency and accuracy, and generalization to seven graph-level tasks, highlighting scalability and robustness across model families and graph domains. By aligning structural and semantic information with target nodes, HS2C offers a principled, scalable path to reliable LLM-based reasoning on large TAGs, reducing input redundancy while preserving critical context.

Abstract

Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding. Recent studies typically focus on verbalizing the graph structures via handcrafted prompts, feeding the target node and its neighborhood context into LLMs. However, constrained by the context window, existing methods mainly resort to random sampling, often implemented via dropping node/edge randomly, which inevitably introduces noise and cause reasoning instability. We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance. To this end, we propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily. Structurally, guided by the principle of Structural Entropy minimization, we perform a global hierarchical partition that decodes the graph's essential topology. This partition identifies naturally cohesive, homophilic communities, while discarding stochastic connectivity noise. Semantically, we deliver the detected structural homophily to the LLM, empowering it to perform differentiated semantic aggregation based on predefined community type. This process compresses redundant background contexts into concise community-level consensus, selectively preserving semantically homophilic information aligned with the target nodes. Extensive experiments on 10 node-level benchmarks across LLMs of varying sizes and families demonstrate that, by feeding LLMs with structurally and semantically compressed inputs, HS2C simultaneously enhances the compression rate and downstream inference accuracy, validating its superiority and scalability. Extensions to 7 diverse graph-level benchmarks further consolidate HS2C's task generalizability.

Figures (17)

• Figure 1: The challenge and our solution. (a) Illustration of LLM reasoning instability induced by random sampling strategy. (b) Illustration of $\text{H}\text{S}_2\text{C}$ compressing the graph to generate compact LLM inputs.
• Figure 2: The overall framework of the proposed $\text{HS}_2\text{C}$, which consists of 3 modules. Firstly, we detect homophilic structures by enhancing the graph topology and minimizing SE to obtain a hierarchical community partition. Secondly, we aggregate the textual attributes of background nodes within each community to generate concise, semantically aligned summaries. Finally, we reconstruct a compressed graph $\widetilde{\mathcal{G}}$ that preserves essential structural and semantic information for downstream inference.
• Figure 3: Comparison of Graph Compression Rate (GCR), classification accuracy (ACC), and data memory (MB).
• Figure 4: Node-level Case study on OGBN‑ArXiv dataset comparing the neighborhood of a target node before and after applying $\text{HS}_2\text{C}$. The left side shows the dense neighbors of target node in the original graph $\mathcal{G}$. Right: the compressed neighbors after homophilic structure detection and semantic aggregation in $\widetilde{\mathcal{G}}$.
• Figure 5: Ablation study results for $\text{HS}_2\text{C}$'s 3 key components.

Theorems & Definitions (2)

• Definition 1: MERGE Operation
• Definition 2: DROP Operation