Shall Your Data Strategy Work? Perform a Swift Study
Minlong Peng, Jingyi Yang, Zhongjun He, Hua Wu
TL;DR
Shall Your Data Strategy Work? Perform a Swift Study proposes a gradient-based method to rapidly evaluate instruction-tuning data strategies without retraining large language models. It introduces a relative influence score $RelInf$ and context-dependent inflows $s_{in}$ and $s_{cross}$ to assess how probe data affect evaluation tasks across model states. The authors apply this framework to three data creation strategies—Chain-of-Thought (CoT), query clarification, and response evaluation—and validate the findings with full retraining experiments on Chinese-LLaMA2 variants, finding consistent alignment. The results highlight that CoT data generally enhances cross-task generalization, while query clarification and response evaluation offer in-task advantages and cross-task benefits, offering a practical, resource-efficient approach for data-strategy design.
Abstract
This work presents a swift method to assess the efficacy of particular types of instruction-tuning data, utilizing just a handful of probe examples and eliminating the need for model retraining. This method employs the idea of gradient-based data influence estimation, analyzing the gradient projections of probe examples from the chosen strategy onto evaluation examples to assess its advantages. Building upon this method, we conducted three swift studies to investigate the potential of Chain-of-thought (CoT) data, query clarification data, and response evaluation data in enhancing model generalization. Subsequently, we embarked on a validation study to corroborate the findings of these swift studies. In this validation study, we developed training datasets tailored to each studied strategy and compared model performance with and without the use of these datasets. The results of the validation study aligned with the findings of the swift studies, validating the efficacy of our proposed method.