Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature

Devvrat Joshi, Islem Rekik

Abstract

Automated knowledge graph (KG) construction is essential for navigating the rapidly expanding body of scientific literature. However, existing approaches struggle to recognize long multi-word entities, often fail to generalize across domains, and typically overlook the hierarchical nature of scientific knowledge. While general-purpose large language models (LLMs) offer adaptability, they are computationally expensive and yield inconsistent accuracy on specialized tasks. As a result, current KGs are shallow and inconsistent, limiting their utility for exploration and synthesis. We propose a two-stage framework for scalable, zero-shot scientific KG construction. The first stage, Z-NERD, introduces (i) Orthogonal Semantic Decomposition (OSD), which promotes domain-agnostic entity recognition by isolating semantic "turns" in text, and (ii) a Multi-Scale TCQK attention mechanism that captures coherent multi-word entities through n-gram-aware attention heads. The second stage, HGNet, performs relation extraction with hierarchy-aware message passing, explicitly modeling parent, child, and peer relations. To enforce global consistency, we introduce two complementary objectives: a Differentiable Hierarchy Loss to discourage cycles and shortcut edges, and a Continuum Abstraction Field (CAF) Loss that embeds abstraction levels along a learnable axis in Euclidean space. This is the first approach to formalize hierarchical abstraction as a continuous property within standard Euclidean embeddings, offering a simpler alternative to hyperbolic methods. We release SPHERE (https://github.com/basiralab/SPHERE), a multi-domain benchmark for hierarchical relation extraction. Our framework establishes a new state of the art on SciERC, SciER, and SPHERE, improving NER by 8.08% and RE by 5.99% on out-of-distribution tests. In zero-shot settings, gains reach 10.76% for NER and 26.2% for RE.

HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature

Abstract

Automated knowledge graph (KG) construction is essential for navigating the rapidly expanding body of scientific literature. However, existing approaches struggle to recognize long multi-word entities, often fail to generalize across domains, and typically overlook the hierarchical nature of scientific knowledge. While general-purpose large language models (LLMs) offer adaptability, they are computationally expensive and yield inconsistent accuracy on specialized tasks. As a result, current KGs are shallow and inconsistent, limiting their utility for exploration and synthesis. We propose a two-stage framework for scalable, zero-shot scientific KG construction. The first stage, Z-NERD, introduces (i) Orthogonal Semantic Decomposition (OSD), which promotes domain-agnostic entity recognition by isolating semantic "turns" in text, and (ii) a Multi-Scale TCQK attention mechanism that captures coherent multi-word entities through n-gram-aware attention heads. The second stage, HGNet, performs relation extraction with hierarchy-aware message passing, explicitly modeling parent, child, and peer relations. To enforce global consistency, we introduce two complementary objectives: a Differentiable Hierarchy Loss to discourage cycles and shortcut edges, and a Continuum Abstraction Field (CAF) Loss that embeds abstraction levels along a learnable axis in Euclidean space. This is the first approach to formalize hierarchical abstraction as a continuous property within standard Euclidean embeddings, offering a simpler alternative to hyperbolic methods. We release SPHERE (https://github.com/basiralab/SPHERE), a multi-domain benchmark for hierarchical relation extraction. Our framework establishes a new state of the art on SciERC, SciER, and SPHERE, improving NER by 8.08% and RE by 5.99% on out-of-distribution tests. In zero-shot settings, gains reach 10.76% for NER and 26.2% for RE.
Paper Structure (60 sections, 14 equations, 7 figures, 11 tables)

This paper contains 60 sections, 14 equations, 7 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Entity Recognition in Scientific Text
  4. Relation Extraction and Hierarchical Modeling
  5. Geometric and Logical Representations of Hierarchy
  6. Methodology
  7. Z-NERD: Zero-Shot Entity Recognition
  8. Domain Generalization via Orthogonal Semantic Decomposition (OSD)
  9. Coherent Multi-Word Entities via Multi-Scale TCQK Attention
  10. HGNet: Hierarchy Graph Network
  11. Probabilistic Hierarchical Message Passing
  12. Logical Coherence via Differentiable Hierarchy Loss (DHL)
  13. Geometric Coherence via Continuum Abstraction Field (CAF) Loss
  14. Final Relation Prediction
  15. Coherent Architecture and Joint Optimization
  16. ...and 45 more sections

Figures (7)

  • Figure 1: Continuous axis of abstraction for topics in physics.
  • Figure 2: Average Orthogonal Semantic Velocity for tokens at entity boundaries ('Start'/'End') vs. 'Non-entity' tokens for SPHERE-CS (left) and SciER (right). The clear separation provides visual evidence for Hypothesis 3.1.
  • Figure 3: Distribution of abstraction scores across different scientific domains, showing distinct patterns for each field.
  • Figure 4: Example of physics domain hierarchical knowledge graph. Hierarchy radially extends outward.
  • Figure 5: Loss plot of HGNet on computer science SPHERE dataset.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Definition A.1: Abstraction Field Vector