Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

Yang Xiang, Li Fan, Chenke Yin, Chengtao Ji

TL;DR

This work introduces QUIET, a Quantized Hierarchical Self-weighted Tokenizer, to provide a task-adaptive discrete token interface for Graph Foundation Models. By combining hierarchical residual vector quantization with a light-weight, self-weighted gating mechanism, QUIET exposes multi-scale graph tokens and dynamically adjusts their contribution per downstream task while keeping the backbone encoder frozen. Empirical results on node classification and link prediction show consistent gains over strong baselines under matched compute, and ablations confirm the importance of gating, codebook diversity, and balancing. The approach offers a scalable, efficient path toward unified graph pre-training and deployment across diverse graph tasks.

Abstract

Recent progress in language and vision foundation models demonstrates the importance of discrete token interfaces that transform complex inputs into compact sequences for large-scale modeling. Extending this paradigm to graphs requires a tokenization scheme that handles non-Euclidean structures and multi-scale dependencies efficiently. Existing approaches to graph tokenization, linearized, continuous, and quantized, remain limited in adaptability and efficiency. In particular, most current quantization-based tokenizers organize hierarchical information in fixed or task-agnostic ways, which may either over-represent or under-utilize structural cues, and lack the ability to dynamically reweight contributions from different levels without retraining the encoder. This work presents a hierarchical quantization framework that introduces a self-weighted mechanism for task-adaptive aggregation across multiple scales. The proposed method maintains a frozen encoder while modulating information flow through a lightweight gating process, enabling parameter-efficient adaptation to diverse downstream tasks. Experiments on benchmark datasets for node classification and link prediction demonstrate consistent improvements over strong baselines under comparable computational budgets.

A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

TL;DR

This work introduces QUIET, a Quantized Hierarchical Self-weighted Tokenizer, to provide a task-adaptive discrete token interface for Graph Foundation Models. By combining hierarchical residual vector quantization with a light-weight, self-weighted gating mechanism, QUIET exposes multi-scale graph tokens and dynamically adjusts their contribution per downstream task while keeping the backbone encoder frozen. Empirical results on node classification and link prediction show consistent gains over strong baselines under matched compute, and ablations confirm the importance of gating, codebook diversity, and balancing. The approach offers a scalable, efficient path toward unified graph pre-training and deployment across diverse graph tasks.

Abstract

Recent progress in language and vision foundation models demonstrates the importance of discrete token interfaces that transform complex inputs into compact sequences for large-scale modeling. Extending this paradigm to graphs requires a tokenization scheme that handles non-Euclidean structures and multi-scale dependencies efficiently. Existing approaches to graph tokenization, linearized, continuous, and quantized, remain limited in adaptability and efficiency. In particular, most current quantization-based tokenizers organize hierarchical information in fixed or task-agnostic ways, which may either over-represent or under-utilize structural cues, and lack the ability to dynamically reweight contributions from different levels without retraining the encoder. This work presents a hierarchical quantization framework that introduces a self-weighted mechanism for task-adaptive aggregation across multiple scales. The proposed method maintains a frozen encoder while modulating information flow through a lightweight gating process, enabling parameter-efficient adaptation to diverse downstream tasks. Experiments on benchmark datasets for node classification and link prediction demonstrate consistent improvements over strong baselines under comparable computational budgets.
Paper Structure (18 sections, 15 equations, 2 figures, 4 tables)

This paper contains 18 sections, 15 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Graph Foundation Models
  4. Graph Tokenization
  5. Preliminary
  6. Methodology
  7. Quantized Hierarchical Self-weighted Tokenizer
  8. Node Encoding
  9. Hierarchical Residual Vector Quantization
  10. Task-adaptive Self-weighted Gate
  11. Graph Foundation Model
  12. Training Objective
  13. Experiments
  14. Experimental Setup
  15. Overall Performance
  16. ...and 3 more sections

Figures (2)

  • Figure 1: The overall framework of QUIET.
  • Figure 2: Ablation study of different components in our proposed QUIET on Cora, Citeseer, and Pubmed datasets for node classification (left) and link prediction (right) tasks.