GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks

TL;DR

GraphTranslator addresses the limitation that traditional graph models are restricted to predefined tasks and do not support open-ended, instruction-guided reasoning. By introducing a Translator to map node embeddings to language token space and a Producer to generate alignment data, the framework enables a frozen GM to handle predefined tasks while an LLM provides open-ended capabilities; two-stage training aligns embeddings and language representations. Experiments show improved performance on zero-shot node classification and promising graph question answering with instruction-based prompts, demonstrating practical potential for interactive graph analytics. The work advances scalable, explainable graph reasoning by integrating structured graph representations with natural-language interfaces.

Abstract

Large language models (LLMs) like ChatGPT, exhibit powerful zero-shot and instruction-following capabilities, have catalyzed a revolutionary transformation across diverse fields, especially for open-ended tasks. While the idea is less explored in the graph domain, despite the availability of numerous powerful graph models (GMs), they are restricted to tasks in a pre-defined form. Although several methods applying LLMs to graphs have been proposed, they fail to simultaneously handle the pre-defined and open-ended tasks, with LLM as a node feature enhancer or as a standalone predictor. To break this dilemma, we propose to bridge the pretrained GM and LLM by a Translator, named GraphTranslator, aiming to leverage GM to handle the pre-defined tasks effectively and utilize the extended interface of LLMs to offer various open-ended tasks for GM. To train such Translator, we propose a Producer capable of constructing the graph-text alignment data along node information, neighbor information and model information. By translating node representation into tokens, GraphTranslator empowers an LLM to make predictions based on language instructions, providing a unified perspective for both pre-defined and open-ended tasks. Extensive results demonstrate the effectiveness of our proposed GraphTranslator on zero-shot node classification. The graph question answering experiments reveal our GraphTranslator potential across a broad spectrum of open-ended tasks through language instructions. Our code is available at: https://github.com/alibaba/GraphTranslator.

Figures (4)

• Figure 1: Intuitive illustration of our GraphTranslator (a) Comparisons of GraphTranslator with popular paradigms of applying LLMs to graphs. Unlike using LLM as enhancer or sole predictor, GraphTranslator bridges LLM and GM, handling both pre-defined and open-ended tasks. (b) Demonstrations of tasks in GraphTranslator, where GM is leveraged for pre-defined tasks, and the LLM is extended as the interface of GM for open-ended tasks.
• Figure 2: The overall framework of our GraphTranslator, which aligns GM to LLM by Translator for open-ended tasks. We train the lightweight Translator module following a two-stage paradigm, with the alignment data generated by our Producer.
• Figure 3: Quantitative analysis of graph question answering. The order of response quality ranking is as follows: A > B > C > D.
• Figure 4: A case of graph question answering on Taobao Dataset.