The Harder The Better: Maintaining Supervised Fine-tuning Generalization with Less but Harder Data
Zhaoyang Shang, Sibo Wei, Jianbin Guo, Rui Zhou, Lifeng Dong, Yin Luo
TL;DR
This work introduces THTB, a Bloom's Taxonomy-inspired framework for data selection and annotation guidance in supervised fine-tuning. It integrates quality filtering with intrinsic hardness (Bloom Score and Interdisciplinary Complexity) and extrinsic hardness (IREI and Silhouette Coefficient) to prioritize higher-level cognitive instructions. The three-stage pipeline enables substantial data efficiency, with 5% of data sometimes surpassing full-data training and 2% sufficing for strong vertical-domain annotation guidance, as demonstrated in Traditional Chinese Medicine. The approach yields improved generalization and domain adaptation capabilities, supported by quantitative gains across MMLU-Pro and Alpaca-Eval, and provides interpretable criteria for data construction and annotation strategies.
Abstract
Large Language Models (LLMs) excel in general tasks, but adapting them to specialized domains relies on high-quality supervised fine-tuning (SFT) data. Although existing methods can identify subsets of high-quality data and reduce training cost to some extent, their selection process still suffers from over-reliance on LLMs' internal knowledge, weak interpretability, and limited generalization. To address these limitations, we propose THTB (The Harder The Better), a cognitive science-inspired framework for instruction data selection and annotation guidance. THTB prioritizes higher-level cognitive instructions by combining quality filtering with intrinsic and extrinsic hardness scoring, offering interpretable and quantifiable criteria for efficient SFT, both in data selection and annotation guidance. Experiments show that THTB enables models trained on only 5% of the data to outperform full-dataset training, while achieving superior generalization compared with LLM-only selection. In addition, THTB provides effective annotation guidance in vertical domains, enabling a model trained on just 2% of the data to surpass models trained on much larger datasets, demonstrating strong potential for domain adaptation. Our code, datasets, and models are available on https://github.com/DYJG-research/THTB.