Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs

Yifan Wei, Xiaoyan Yu, Tengfei Pan, Angsheng Li, Li Du

TL;DR

SENATOR tackles knowledge deficiencies in domain-specific LLMs by coupling Structure Entropy with a knowledge-graph–driven MCTS to identify uncertain knowledge paths. It then synthesizes targeted QA data along these paths to perform supervised fine-tuning, achieving notable gains on medical benchmarks for Llama-3-8B and Qwen2-7B with relatively small synthetic datasets. The approach demonstrates expanded coverage beyond pretraining data and exhibits model- and domain-specific deficiency patterns, suggesting synthetic, deficiency-targeted data as an efficient fuel for continual LLM improvement. This work highlights practical impact for high-stakes domains by improving factual accuracy while outlining avenues for data quality, KG dependency, and scaling considerations.

Abstract

Large language models (LLMs) have achieved unprecedented performance by leveraging vast pretraining corpora, yet their performance remains suboptimal in knowledge-intensive domains such as medicine and scientific research, where high factual precision is required. While synthetic data provides a promising avenue for augmenting domain knowledge, existing methods frequently generate redundant samples that do not align with the model's true knowledge gaps. To overcome this limitation, we propose a novel Structural Entropy-guided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs. Our approach employs the Structure Entropy (SE) metric to quantify uncertainty along knowledge graph paths and leverages Monte Carlo Tree Search (MCTS) to selectively explore regions where the model lacks domain-specific knowledge. Guided by these insights, the framework generates targeted synthetic data for supervised fine-tuning, enabling continuous self-improvement. Experimental results on LLaMA-3 and Qwen2 across multiple domain-specific benchmarks show that SENATOR effectively detects and repairs knowledge deficiencies, achieving notable performance improvements. The code and data for our methods and experiments are available at https://github.com/weiyifan1023/senator.

Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs

TL;DR

SENATOR tackles knowledge deficiencies in domain-specific LLMs by coupling Structure Entropy with a knowledge-graph–driven MCTS to identify uncertain knowledge paths. It then synthesizes targeted QA data along these paths to perform supervised fine-tuning, achieving notable gains on medical benchmarks for Llama-3-8B and Qwen2-7B with relatively small synthetic datasets. The approach demonstrates expanded coverage beyond pretraining data and exhibits model- and domain-specific deficiency patterns, suggesting synthetic, deficiency-targeted data as an efficient fuel for continual LLM improvement. This work highlights practical impact for high-stakes domains by improving factual accuracy while outlining avenues for data quality, KG dependency, and scaling considerations.

Abstract

Large language models (LLMs) have achieved unprecedented performance by leveraging vast pretraining corpora, yet their performance remains suboptimal in knowledge-intensive domains such as medicine and scientific research, where high factual precision is required. While synthetic data provides a promising avenue for augmenting domain knowledge, existing methods frequently generate redundant samples that do not align with the model's true knowledge gaps. To overcome this limitation, we propose a novel Structural Entropy-guided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs. Our approach employs the Structure Entropy (SE) metric to quantify uncertainty along knowledge graph paths and leverages Monte Carlo Tree Search (MCTS) to selectively explore regions where the model lacks domain-specific knowledge. Guided by these insights, the framework generates targeted synthetic data for supervised fine-tuning, enabling continuous self-improvement. Experimental results on LLaMA-3 and Qwen2 across multiple domain-specific benchmarks show that SENATOR effectively detects and repairs knowledge deficiencies, achieving notable performance improvements. The code and data for our methods and experiments are available at https://github.com/weiyifan1023/senator.
Paper Structure (34 sections, 7 equations, 13 figures, 2 tables)

This paper contains 34 sections, 7 equations, 13 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Knowledge Deficiency Detection of LLMs
  4. Model Self-Improvement
  5. Methodology
  6. Structural Entropy Guided Knowledge Deficiency Detection
  7. Self-Information for Measuring Triplet-Level Uncertainty
  8. Structural Entropy of Modeling Knowledge Path-Level Uncertainty
  9. MCTS for Knowledge Deficiency Detection
  10. Node Selection.
  11. Path Expansion.
  12. Reward Estimation.
  13. Backpropagation.
  14. Deficiency Knowledge Synthesis and Repair
  15. Experiments
  16. ...and 19 more sections

Figures (13)

  • Figure 1: The SENATOR framework operates as follows: An entity state in the knowledge graph is (a) selected, (b) expanded, and (c) simulated using the LLM agent until a terminal node is reached. Specifically, we employ a random policy $\pi$ during the expansion phase. (d) Subsequently, signals from the value function $V(\cdot)$ are backpropagated. This process is iterated multiple times, with the MCTS algorithm searching for (f) better trajectories guided by (e) signals from structural entropy to (g) generate data addressing knowledge deficiencies, (h) and repair model knowledge.
  • Figure 2: Distribution of Pretraining Corpus vs. Synthetic Data. In (a)-(d), blue regions represent the medical pretraining corpus (PubMedQA, MedQA, MedMCQA, and their hybrid), red regions show synthetic data generated by Llama-3. In (e)-(h), red regions indicate synthetic data produced by Qwen2. Darker areas reflect higher concentrations of data points, lighter areas vice versa.
  • Figure 3: Distribution of Data Generated by Llama-3 (red) and Qwen2 (blue).
  • Figure 4: Performance differences for various data compositions.
  • Figure 5: Example prompt for the synthetic data generation stage of SENATOR.
  • ...and 8 more figures