H4G: Unlocking Faithful Inference for Zero-Shot Graph Learning in Hyperbolic Space

TL;DR

This work proposes H4G, a framework that systematically reduces embedding radii using learnable block-diagonal scaling matrices and Mobius matrix multiplication, and demonstrates that radius reduction enables faithful multi-scale representation for advancing zero-shot graph learning.

Abstract

Text-attributed graphs are widely used across domains, offering rich opportunities for zero-shot learning via graph-text alignment. However, existing methods struggle with tasks requiring fine-grained pattern recognition, particularly on heterophilic graphs. Through empirical and theoretical analysis, we identify an \textbf{over-abstraction problem}: current approaches operate at excessively large hyperbolic radii, compressing multi-scale structural information into uniform high-level abstractions. This abstraction-induced information loss obscures critical local patterns essential for accurate predictions. By analyzing embeddings in hyperbolic space, we demonstrate that optimal graph learning requires \textbf{faithful preservation} of fine-grained structural details, better retained by representations positioned closer to the origin. To address this, we propose \textbf{H4G}, a framework that systematically reduces embedding radii using learnable block-diagonal scaling matrices and Möbius matrix multiplication. This approach restores access to fine-grained patterns while maintaining global receptive ability with minimal computational overhead. Experiments show H4G achieves state-of-the-art zero-shot performance with \textbf{12.8\%} improvement on heterophilic graphs and \textbf{8.4\%} on homophilic graphs, confirming that radius reduction enables faithful multi-scale representation for advancing zero-shot graph learning.

Figures (5)

• Figure 1: Three main categories for Integrating Graphs with Large Language Models.
• Figure 2: Hyperbolic radius distribution of text embeddings. Baseline methods operate at large radii with dispersed distributions, losing fine-grained structural information. H4G achieves systematic radius reduction with concentrated embeddings, preserving detailed graph patterns essential for effective zero-shot learning.
• Figure 3: Overview of the H4G Framework. H4G achieves fine-grained representation learning through radius adjustment in hyperbolic space. Text-attributed graphs are encoded by graph and text models and then projected into the Poincaré ball of hyperbolic space. H4G employs learnable block-diagonal scaling matrices with Möbius matrix multiplication to systematically adjust embeddings from high abstraction levels (far from origin) to fine-grained information levels (close to origin), preserving richer structural details in the representation space for high-quality graph-text alignment in zero-shot learning.
• Figure 4: Ablation study on hyperbolic projection strategies across four datasets. "Text only" applies hyperbolic projection solely to text embeddings while keeping graph embeddings in Euclidean space. "Graph only" projects only graph representations into hyperbolic space. "Both" represents our full H4G framework with dual hyperbolic projection. Results demonstrate that projecting both modalities into hyperbolic space achieves optimal alignment quality and superior zero-shot transfer performance.
• Figure 5: Sensitivity analysis of block size and curvature on zero-shot node classification accuracy. The surface demonstrates that H4G achieves optimal performance when block size equals 32 and curvature equals 1.0, validating the effectiveness of fine-grained radius control in appropriately curved hyperbolic space. Performance degrades when block size becomes too small or too large, or when curvature deviates significantly from the optimal value.