NumCoKE: Ordinal-Aware Numerical Reasoning over Knowledge Graphs with Mixture-of-Experts and Contrastive Learning

TL;DR

NumCoKE tackles the problem of numerical reasoning on knowledge graphs by integrating entities, relations, and numeric attributes into a unified, context-aware representation. It introduces a Mixture-of-Experts Knowledge-Aware encoder (MoEKA) that dynamically routes numeric attributes to relation-specific experts, and ordinal knowledge contrastive learning (OKCL) to generate ordinal-aware samples for fine-grained value discrimination. The method achieves state-of-the-art results on three public benchmarks (US-Cities, Spotify, Credit) and demonstrates robustness across discrete, continuous, and skewed attribute distributions, with notable gains on the Credit dataset. Overall, NumCoKE advances numerical reasoning in KGs by coupling semantic integration with ordinal discrimination, enabling more accurate quantitative inference for downstream AI tasks.

Abstract

Knowledge graphs (KGs) serve as a vital backbone for a wide range of AI applications, including natural language understanding and recommendation. A promising yet underexplored direction is numerical reasoning over KGs, which involves inferring new facts by leveraging not only symbolic triples but also numerical attribute values (e.g., length, weight). However, existing methods fall short in two key aspects: (1) Incomplete semantic integration: Most models struggle to jointly encode entities, relations, and numerical attributes in a unified representation space, limiting their ability to extract relation-aware semantics from numeric information. (2) Ordinal indistinguishability: Due to subtle differences between close values and sampling imbalance, models often fail to capture fine-grained ordinal relationships (e.g., longer, heavier), especially in the presence of hard negatives. To address these challenges, we propose NumCoKE, a numerical reasoning framework for KGs based on Mixture-of-Experts and Ordinal Contrastive Embedding. To overcome (C1), we introduce a Mixture-of-Experts Knowledge-Aware (MoEKA) encoder that jointly aligns symbolic and numeric components into a shared semantic space, while dynamically routing attribute features to relation-specific experts. To handle (C2), we propose Ordinal Knowledge Contrastive Learning (OKCL), which constructs ordinal-aware positive and negative samples using prior knowledge, enabling the model to better discriminate subtle semantic shifts. Extensive experiments on three public KG benchmarks demonstrate that NumCoKE consistently outperforms competitive baselines across diverse attribute distributions, validating its superiority in both semantic integration and ordinal reasoning.

Figures (9)

• Figure 1: Example of KG-based numerical reasoning. Same colored text indicates strong related relation and attribute.
• Figure 2: The Overview of our model. (a) The MoEKA Encoder encodes each entity with the relation and attributes to a unified, elaborate semantic representation. (b) To capture the fine-grained semantic information, we utilize a new knowledge contrastive learning method to generate high-quality ordinal samples to learn the nuances in attributes and distinguish similar semantics.
• Figure 3: Research of Hyperparameter K on Spotify dataset.
• Figure 4: Proportional test on Credit and Spotify. A relation percentage of 100% means that we only consider relations, while 0% means that we only consider entities.
• Figure 5: Visualization of relevance scores of each numeric attribute of RAKGE and NumCoKE on Credit dataset.