Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kastor: Fine-tuned Small Language Models for Shape-based Active Relation Extraction

Ringwald Celian, Gandon Fabien, Faron Catherine, Michel Franck, Abi Akl Hanna

TL;DR

Kastor presents a shape-based RDF relation extraction framework that fine-tunes small language models on noisy, domain-specific KBs by leveraging SHACL patterns. It relaxes the traditional maximal-shape constraint to use example-specific patterns and applies rule-based graph augmentation and knowledge distillation to build a refined training base. An iterative, light active learning loop with a domain expert generates gold data, enabling the SLM to generalize to unseen patterns and improve KB completion. The approach demonstrates strong per-sample pattern diversity, affordable training cost, and improved factual coverage, suggesting practical impact for scalable, domain-focused KB curation.

Abstract

RDF pattern-based extraction is a compelling approach for fine-tuning small language models (SLMs) by focusing a relation extraction task on a specified SHACL shape. This technique enables the development of efficient models trained on limited text and RDF data. In this article, we introduce Kastor, a framework that advances this approach to meet the demands for completing and refining knowledge bases in specialized domains. Kastor reformulates the traditional validation task, shifting from single SHACL shape validation to evaluating all possible combinations of properties derived from the shape. By selecting the optimal combination for each training example, the framework significantly enhances model generalization and performance. Additionally, Kastor employs an iterative learning process to refine noisy knowledge bases, enabling the creation of robust models capable of uncovering new, relevant facts

Kastor: Fine-tuned Small Language Models for Shape-based Active Relation Extraction

TL;DR

Kastor presents a shape-based RDF relation extraction framework that fine-tunes small language models on noisy, domain-specific KBs by leveraging SHACL patterns. It relaxes the traditional maximal-shape constraint to use example-specific patterns and applies rule-based graph augmentation and knowledge distillation to build a refined training base. An iterative, light active learning loop with a domain expert generates gold data, enabling the SLM to generalize to unseen patterns and improve KB completion. The approach demonstrates strong per-sample pattern diversity, affordable training cost, and improved factual coverage, suggesting practical impact for scalable, domain-focused KB curation.

Abstract

RDF pattern-based extraction is a compelling approach for fine-tuning small language models (SLMs) by focusing a relation extraction task on a specified SHACL shape. This technique enables the development of efficient models trained on limited text and RDF data. In this article, we introduce Kastor, a framework that advances this approach to meet the demands for completing and refining knowledge bases in specialized domains. Kastor reformulates the traditional validation task, shifting from single SHACL shape validation to evaluating all possible combinations of properties derived from the shape. By selecting the optimal combination for each training example, the framework significantly enhances model generalization and performance. Additionally, Kastor employs an iterative learning process to refine noisy knowledge bases, enabling the creation of robust models capable of uncovering new, relevant facts

Paper Structure

This paper contains 27 sections, 21 equations, 6 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Kastor framework
  4. The original task: an extraction focused on a maximal target shape $s^*$
  5. Example-specific patterns and Rule-based graph augmentation
  6. Knowledge distillation: from $\mathcal{K}$ to $\mathcal{K}^{\mathcal{WR}\models}_{\mathbb{P}(s^*)}$
  7. An iterative SLM learning process with human in the loop
  8. Learning from a small dataset.
  9. The sampler.
  10. SLM training details.
  11. Light active learning.
  12. Cross-evaluation of the models.
  13. Experimental results
  14. Resulting Datasets
  15. Models performances
  16. ...and 12 more sections

Figures (6)

  • Figure 1: Knowledge distillation: from the initial base to a refined version based on shape $s^*$
  • Figure 2: $\mathbb{P}_{\mathcal{K}}(s^*)$ Inverse cumulative frequency distribution
  • Figure 3: 10 most frequent $\pi \in \mathbb{P}_{\mathcal{K}}(s^*)$
  • Figure 4: Overview the experimental set-up including the iterative SLM light active learning process
  • Figure 5: Errors distribution over the different models and datasets
  • ...and 1 more figures