Geometric Structural Knowledge Graph Foundation Model

TL;DR

Gamma tackles the limitation of single geometric transformations in inductive KG reasoning by introducing multi-head geometric attention that runs parallel branches in real, complex, split-complex, and dual-number spaces. An adaptive, entropy-regularized fusion mechanism weighted by a query context combines branch messages, yielding richer relational representations and stronger zero-shot generalization. Across 56 diverse KGs, Gamma consistently outperforms the prior Ultra baseline, especially in inductive scenarios, with notable gains from the complementary biases of different geometric spaces. The work provides a formal framework, extensive ablations, and practical insights into how multi-branch, attention-guided fusion enhances expressiveness and generalization for structural KG foundation models.

Abstract

Structural knowledge graph foundation models aim to generalize reasoning to completely new graphs with unseen entities and relations. A key limitation of existing approaches like Ultra is their reliance on a single relational transformation (e.g., element-wise multiplication) in message passing, which can constrain expressiveness and fail to capture diverse relational and structural patterns exhibited on diverse graphs. In this paper, we propose Gamma, a novel foundation model that introduces multi-head geometric attention to knowledge graph reasoning. Gamma replaces the single relational transformation with multiple parallel ones, including real, complex, split-complex, and dual number based transformations, each designed to model different relational structures. A relational conditioned attention fusion mechanism then adaptively fuses them at link level via a lightweight gating with entropy regularization, allowing the model to robustly emphasize the most appropriate relational bias for each triple pattern. We present a full formalization of these algebraic message functions and discuss how their combination increases expressiveness beyond any single space. Comprehensive experiments on 56 diverse knowledge graphs demonstrate that Gamma consistently outperforms Ultra in zero-shot inductive link prediction, with a 5.5% improvement in mean reciprocal rank on the inductive benchmarks and a 4.4% improvement across all benchmarks, highlighting benefits from complementary geometric representations.

Figures (1)

• Figure 1: Overall architecture of the Gamma model.