Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniGTE: Unified Graph-Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains

Duo Wang, Yuan Zuo, Guangyue Lu, Junjie Wu

TL;DR

UniGTE addresses zero-shot generalization across graph tasks by unifying graph structure and natural-language task semantics in a single encoder–decoder. It uses a structure-aware, RoPE-enabled attention mechanism and learnable alignment tokens to produce a task-aware graph representation that conditions a frozen LLM decoder to both predict task outputs and reconstruct the input prompt. Trained in an instruction-tuning regime on diverse node-, edge-, and graph-level datasets across multiple domains, UniGTE achieves state-of-the-art zero-shot performance in node classification, link prediction, graph classification, and graph regression under cross-task and cross-domain settings. This tight integration of structural cues with LLM semantics enables robust, transferable graph reasoning without task-specific fine-tuning, with strong implications for scalable, multi-task graph learning.

Abstract

Generalizing to unseen graph tasks without task-specific supervision is challenging: conventional graph neural networks are typically tied to a fixed label space, while large language models (LLMs) struggle to capture graph structure. We introduce UniGTE, an instruction-tuned encoder-decoder framework that unifies structural and semantic reasoning. The encoder augments a pretrained autoregressive LLM with learnable alignment tokens and a structure-aware graph-text attention mechanism, enabling it to attend jointly to a tokenized graph and a natural-language task prompt while remaining permutation-invariant to node order. This yields compact, task-aware graph representations. Conditioned solely on these representations, a frozen LLM decoder predicts and reconstructs: it outputs the task answer and simultaneously paraphrases the input graph in natural language. The reconstruction objective regularizes the encoder to preserve structural cues. UniGTE is instruction-tuned on five datasets spanning node-level, edge-level, and graph-level tasks across diverse domains, yet requires no fine-tuning at inference. It achieves new state-of-the-art zero-shot results on node classification, link prediction, graph classification, and graph regression under cross-task and cross-domain settings, demonstrating that tight integration of graph structure with LLM semantics enables robust, transferable graph reasoning.

UniGTE: Unified Graph-Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains

TL;DR

UniGTE addresses zero-shot generalization across graph tasks by unifying graph structure and natural-language task semantics in a single encoder–decoder. It uses a structure-aware, RoPE-enabled attention mechanism and learnable alignment tokens to produce a task-aware graph representation that conditions a frozen LLM decoder to both predict task outputs and reconstruct the input prompt. Trained in an instruction-tuning regime on diverse node-, edge-, and graph-level datasets across multiple domains, UniGTE achieves state-of-the-art zero-shot performance in node classification, link prediction, graph classification, and graph regression under cross-task and cross-domain settings. This tight integration of structural cues with LLM semantics enables robust, transferable graph reasoning without task-specific fine-tuning, with strong implications for scalable, multi-task graph learning.

Abstract

Generalizing to unseen graph tasks without task-specific supervision is challenging: conventional graph neural networks are typically tied to a fixed label space, while large language models (LLMs) struggle to capture graph structure. We introduce UniGTE, an instruction-tuned encoder-decoder framework that unifies structural and semantic reasoning. The encoder augments a pretrained autoregressive LLM with learnable alignment tokens and a structure-aware graph-text attention mechanism, enabling it to attend jointly to a tokenized graph and a natural-language task prompt while remaining permutation-invariant to node order. This yields compact, task-aware graph representations. Conditioned solely on these representations, a frozen LLM decoder predicts and reconstructs: it outputs the task answer and simultaneously paraphrases the input graph in natural language. The reconstruction objective regularizes the encoder to preserve structural cues. UniGTE is instruction-tuned on five datasets spanning node-level, edge-level, and graph-level tasks across diverse domains, yet requires no fine-tuning at inference. It achieves new state-of-the-art zero-shot results on node classification, link prediction, graph classification, and graph regression under cross-task and cross-domain settings, demonstrating that tight integration of graph structure with LLM semantics enables robust, transferable graph reasoning.
Paper Structure (60 sections, 12 equations, 2 figures, 14 tables)

This paper contains 60 sections, 12 equations, 2 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Task definition and notation
  4. Unified graph-text encoder: learning task-aware graph representations via LLM
  5. Unified input formatting for graph tasks
  6. Structure-aware graph-text attention
  7. Cross-modal RoPE
  8. Distance-based structural bias
  9. Edge-aware bias
  10. Masking bias
  11. Instruction tuning of the unified graph–text encoder
  12. Instruction-tuning objective.
  13. Auxiliary prompt reconstruction.
  14. Overall objective.
  15. Training and evaluation strategy
  16. ...and 45 more sections

Figures (2)

  • Figure 1: Framework of UniGTE
  • Figure 2: Ablation study of alignment tokens and task-aware graph encoding, reported from both domain-level and task-level perspectives. "w/o AT" indicates the removal of alignment tokens, while "w/o TA" refers to replacing task-specific descriptions with a generic prompt.