Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix

Zile Yang, Ling Li, Na Di, Jinlong Pang, Yao Zhou, Hao Cheng, Bo Han, Jiaheng Wei

TL;DR

This paper tackles the bottleneck of relying on scarce high-quality data for supervised fine-tuning of instruction-following models. It proposes ENTP, a neural-symbolic purge-mix framework that first purges genuinely noisy low-quality data using symbolic priors and score-consensus mechanisms, then enriches the remaining signal with neural reconstruction and a two-to-one corpus fusion guided by domain-aware symbolic prompts. By combining a Score Transition Matrix, consensus vectors, one-hop clustering, and neural-symbolic fusion cycles, ENTP constructs synthetic corpora from low-quality data that outperform 13 baselines and even rival or exceed models trained on the full original dataset across five benchmarks. The results underscore the latent value in low-quality data and demonstrate that intelligent purification and synthesis can enable more efficient instruction alignment and better out-of-distribution generalization in LLM fine-tuning. The approach offers practical implications for scaling SFT with noisy data and provides a blueprint for neural-symbolic data augmentation in instruction-tuning contexts, with empirical scaling laws supporting the benefit of larger ENTP-generated corpora.$

Abstract

Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.

ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix

TL;DR

This paper tackles the bottleneck of relying on scarce high-quality data for supervised fine-tuning of instruction-following models. It proposes ENTP, a neural-symbolic purge-mix framework that first purges genuinely noisy low-quality data using symbolic priors and score-consensus mechanisms, then enriches the remaining signal with neural reconstruction and a two-to-one corpus fusion guided by domain-aware symbolic prompts. By combining a Score Transition Matrix, consensus vectors, one-hop clustering, and neural-symbolic fusion cycles, ENTP constructs synthetic corpora from low-quality data that outperform 13 baselines and even rival or exceed models trained on the full original dataset across five benchmarks. The results underscore the latent value in low-quality data and demonstrate that intelligent purification and synthesis can enable more efficient instruction alignment and better out-of-distribution generalization in LLM fine-tuning. The approach offers practical implications for scaling SFT with noisy data and provides a blueprint for neural-symbolic data augmentation in instruction-tuning contexts, with empirical scaling laws supporting the benefit of larger ENTP-generated corpora.$

Abstract

Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.
Paper Structure (52 sections, 1 theorem, 10 equations, 26 figures, 12 tables, 1 algorithm)

This paper contains 52 sections, 1 theorem, 10 equations, 26 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Purge $\mathbb{+}$ Mix of the Low Quality Data
  3. Preliminary
  4. Score Transition Matrix
  5. ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix
  6. Step 1: Low-Quality Dataset Construction
  7. LLM-Rating Score Correction
  8. Score Transition Matrix & Consensus Vectors
  9. Step 3: Neural‑Symbolic Two-to-One Corpus Fusion
  10. Preparation Step
  11. Cycle 1
  12. Cycle 2
  13. Experiments
  14. Experimental Setup
  15. Source Corpora
  16. ...and 37 more sections

Key Result

Theorem 1

(K-NN Score Clusterability) Sourced Corpora $D$ satisfies K-NN Score Clusterability if $\forall\ n$, the embedding vector $\mathbf{x}_n$ and its k-Nearest Neighbors $\mathbf{x}_{n_1},...,\mathbf{x}_{n_k}$ belong to the same ground-truth class.

Figures (26)

  • Figure 1: Traditional "quality‑first" paradigm (a) v.s. our proposed paradigm (b). Part(a) represents the traditional data-selection paradigm, where only the high-quality data is selected (encircled by a green circle); Part (b) illustrates our proposed paradigm, which exploits information from neglected low-quality corpora to generate more expressive synthetic corpora. Part (c): the legend includes Non-regular circles (corpora with varying degrees of imperfections), Regular circles (larger diameters correspond to more information). Each symbol is color-coded to represent a distinct corpus.
  • Figure 2: Overview of ENTP. Step (1) separates the corpora into two subsets based on corrected LLM scores: high-quality (scores 3–5) and low-quality (scores 0–2); Step (2) clusters the raw low-quality corpora by inter-corpus similarity and then selects the representative corpora for each cluster; Step (3) integrates connectionist and symbolism to fuse corpora through an iterative multi-step process, offering Intra-Cluster Fusion, combining representative corpora within the same cluster, and Inter-Cluster Fusion, merging those from different clusters; yielding hybrid datasets that preserve diversity while enriching informational value.
  • Figure 3: Logic flow of Step 3: all purple blocks represent the connectionist components, corresponding to different LLM-invoking operators, while all orange blocks stand for the symbolic components, involving the utilization of symbolic rules. Step 3 effectively combines the generalization capability of connectionism with the explicit symbolic rule, thereby achieving the purification, and fusion of the low-quality corpus.
  • Figure 4: Neural-Symbolic Two-To-One Corpora Fusion Stepwise Logical Execution Workflow
  • Figure 5: LLM-rating Prompt Template From $\mathrm{DS}^{2}$pang2024improving
  • ...and 21 more figures

Theorems & Definitions (5)

  • Remark
  • Theorem 1
  • Remark
  • Remark
  • Remark