Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm

TL;DR

This work tackles data efficiency in instruction-tuning of large language models by introducing a choice-based subset selection framework that leverages LLMs to evaluate the contribution of option samples rather than scoring individual items. A greedy procedure builds a high-quality, diverse subset incrementally by forming options from the current selection and candidates, guided by a marginal-gain objective and implemented via LLM prompts with windowed context. Empirically, a 9K-sample subset selected by this method can outperform training on the full 52K Alpaca data across multiple models, and scales to large medical datasets, while achieving substantially lower data-access costs (O($K\cdot T_{LLM}$) per selection). The approach delivers strong data efficiency and competitive benchmark results, supported by extensive ablations, efficiency analyses, and a demonstration on medical-domain data. The work suggests a practical, scalable direction for efficient data selection in real-world LLM fine-tuning, with open-source code available for replication.

Abstract

Selecting high-quality and diverse training samples from extensive datasets plays a crucial role in reducing training overhead and enhancing the performance of Large Language Models (LLMs). However, existing studies fall short in assessing the overall value of selected data, focusing primarily on individual quality, and struggle to strike an effective balance between ensuring diversity and minimizing data point traversals. Therefore, this paper introduces a novel choice-based sample selection framework that shifts the focus from evaluating individual sample quality to comparing the contribution value of different samples when incorporated into the subset. Thanks to the advanced language understanding capabilities of LLMs, we utilize LLMs to evaluate the value of each option during the selection process. Furthermore, we design a greedy sampling process where samples are incrementally added to the subset, thereby improving efficiency by eliminating the need for exhaustive traversal of the entire dataset with the limited budget. Extensive experiments demonstrate that selected data from our method not only surpasses the performance of the full dataset but also achieves competitive results with recent powerful studies, while requiring fewer selections. Moreover, we validate our approach on a larger medical dataset, highlighting its practical applicability in real-world applications. Our code and data are available at https://github.com/BIRlz/comperative_sample_selection.

Figures (10)

• Figure 1: Colors represent data categories, while solid / dapple circles respectively stand for high- / low-quality data. (1) Most existing methods adopt a pointwise approach to produce a subset with top-$K$ representative samples but ignore relationships among them. (2) Our method considers the quality and diversity contribution of each sample to the selected subset. For example, although both and exhibit high-quality in candidate set, incorporating into the selected subset is essential for enhancing its diversity, as the current selected subset already contains .
• Figure 2: The prompt employed in our method to select the optimal element from the candidate set $\mathcal{B}$ with the help of LLM, considering response quality and diversity contribution to the set $\mathcal{A}$.
• Figure 3: Comparing models fine-tuned on our method (9K) and full data (52K) on Llama2-7B and Llama2-13B with different LLMs as selector.
• Figure 4: The win score changes with the increasing of data scale by comparing ours with the Full and IFD.
• Figure 5: Comparing our method with AlpaGasus under 9K data on Llama2-7B.