Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning

Hongyi Cai, Jie Li, Mohammad Mahdinur Rahman, Wenzhen Dong

TL;DR

This work tackles data quality bottlenecks in instruction fine-tuning for large language models by introducing Low-Confidence Gold (LCG), a centroid-based, semi-supervised data-filtering framework. LCG pairs centroid-centric pseudo-labeling with an early-stopped classifier to identify low-confidence, informative samples and curate compact, high-quality instruction subsets. Empirical results on Alpaca_52k and open-source LLMs show that 1k–6k curated samples can achieve comparable or superior MT-Bench performance and offer robust gains across multiple benchmarks, with additional improvements when combined with RLHF. The approach generalizes to larger datasets (e.g., WizardLM) and provides a practical, scalable path toward efficient instruction tuning with preserved data diversity and quality.

Abstract

The effectiveness of instruction fine-tuning for Large Language Models is fundamentally constrained by the quality and efficiency of training datasets. This work introduces Low-Confidence Gold (LCG), a novel filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. Through a semi-supervised approach using a lightweight classifier trained on representative samples, LCG curates high-quality subsets while preserving data diversity. Experimental evaluation demonstrates that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods, with substantial improvements on MT-bench and consistent gains across comprehensive evaluation metrics. The framework's efficacy while maintaining model performance establishes a promising direction for efficient instruction tuning.

Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning

TL;DR

This work tackles data quality bottlenecks in instruction fine-tuning for large language models by introducing Low-Confidence Gold (LCG), a centroid-based, semi-supervised data-filtering framework. LCG pairs centroid-centric pseudo-labeling with an early-stopped classifier to identify low-confidence, informative samples and curate compact, high-quality instruction subsets. Empirical results on Alpaca_52k and open-source LLMs show that 1k–6k curated samples can achieve comparable or superior MT-Bench performance and offer robust gains across multiple benchmarks, with additional improvements when combined with RLHF. The approach generalizes to larger datasets (e.g., WizardLM) and provides a practical, scalable path toward efficient instruction tuning with preserved data diversity and quality.

Abstract

The effectiveness of instruction fine-tuning for Large Language Models is fundamentally constrained by the quality and efficiency of training datasets. This work introduces Low-Confidence Gold (LCG), a novel filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. Through a semi-supervised approach using a lightweight classifier trained on representative samples, LCG curates high-quality subsets while preserving data diversity. Experimental evaluation demonstrates that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods, with substantial improvements on MT-bench and consistent gains across comprehensive evaluation metrics. The framework's efficacy while maintaining model performance establishes a promising direction for efficient instruction tuning.

Paper Structure

This paper contains 21 sections, 7 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. K-means Clustering
  4. Problem Setting
  5. Methodology
  6. Motivation
  7. Centroid Coreset Selection for Pseudo-labels
  8. Low-Confidence Gold: Calibrating with Low-confidence samples to select data
  9. Experiments
  10. Experimental Setup
  11. Main Results
  12. Analysis and Discussion
  13. Cross-Dataset Generalization
  14. Ablation Studies
  15. Impact of Confidence Threshold $\tau$
  16. ...and 6 more sections

Figures (4)

  • Figure 1: We target to select complex and quality samples confidence ranking for benefiting LLM training.
  • Figure 2: The overall pipeline of Low-Confidence Gold. We split our pipeline into two main steps: 1) Clustering to get pseudo-labels and centroid data to collect the initial diversity of data. 2) We feed annotated data into a tiny yet effective classifier to rank the confidences for the rest of the distant data to implement subset selection.
  • Figure 3: The data distribution of MultinomialNB across different confidence intervals.
  • Figure 4: The data distribution of DistilBERT across different confidence intervals under various learning rates.