CHAINSFORMER: Numerical Reasoning on Knowledge Graphs from a Chain Perspective

TL;DR

ChainsFormer reframes numerical reasoning on knowledge graphs as chain-based inference over Relation-Attribute Chains (RA-Chains) assembled into a Tree of Chains (ToC). It introduces a Hyperbolic Filter to prune noisy RA-Chains, a Transformer-based Chain Encoder for in-context, step-by-step reasoning, and a Numerically aware affine transfer to adapt chain representations to numeric values. An attention-based Numerical Reasoner weights RA-Chains to produce final predictions, with a min-max regression objective and end-to-end training. Experimental results on FB15K-237 and YAGO15K show substantial improvements over state-of-the-art baselines (up to 20% MAE reduction) and demonstrate the approach’s interpretability through identified key reasoning chains. The work advances scalable, transparent numerical reasoning on KGs and suggests potential integration with multimodal data and larger KG foundations.

Abstract

Reasoning over Knowledge Graphs (KGs) plays a pivotal role in knowledge graph completion or question answering systems, providing richer and more accurate triples and attributes. As numerical attributes become increasingly essential in characterizing entities and relations in KGs, the ability to reason over these attributes has gained significant importance. Existing graph-based methods such as Graph Neural Networks (GNNs) and Knowledge Graph Embeddings (KGEs), primarily focus on aggregating homogeneous local neighbors and implicitly embedding diverse triples. However, these approaches often fail to fully leverage the potential of logical paths within the graph, limiting their effectiveness in exploiting the reasoning process. To address these limitations, we propose ChainsFormer, a novel chain-based framework designed to support numerical reasoning. Chainsformer not only explicitly constructs logical chains but also expands the reasoning depth to multiple hops. Specially, we introduces Relation-Attribute Chains (RA-Chains), a specialized logic chain, to model sequential reasoning patterns. ChainsFormer captures the step-by-step nature of multi-hop reasoning along RA-Chains by employing sequential in-context learning. To mitigate the impact of noisy chains, we propose a hyperbolic affinity scoring mechanism that selects relevant logic chains in a variable-resolution space. Furthermore, ChainsFormer incorporates an attention-based numerical reasoner to identify critical reasoning paths, enhancing both reasoning accuracy and transparency. Experimental results demonstrate that ChainsFormer significantly outperforms state-of-the-art methods, achieving up to a 20.0% improvement in performance. The implementations are available at https://github.com/zhaodazhuang2333/ChainsFormer.

Figures (8)

• Figure 1: An example of numerical reasoning: predicting missing numerical attributes (e.g., birth date) using relations and known attributes in a KG. Our exploration shifts from a graph-based view to a chain-based perspective for numerical reasoning.
• Figure 2: Average number of logic chains connected to each query in YAGO15K and FB15K datasets. The number of logic chains increases significantly with hops, reaching 3.24e+5 for YAGO15K and 3.10e+6 for FB15K at three hops, underscoring the challenges for large-scale, long-distance reasoning.
• Figure 3: The overall architecture of ChainsFormer is shown in (A). ChainsFormer has for key components: Query Retrieval, Hyperbolic Filter, Chain Encoder and Numerical Reasoner. The Chain Encoder, detailed in (B), employs Transformer to represent RA-Chains. The Numerical Reasoner, detailed in (C), utilizes the proposed attention mechanism to weight and reason over the chains.
• Figure 4: Performance comparison across different reasoning settings. The results are evaluated using Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) on the YAGO15K and FB15K-237 datasets.
• Figure 5: A case study for ChainsFormer's reasoning process on Francis Ford Coppola's birth date.