Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection

TL;DR

G2IS introduces a gradient-based instruction graph to capture the joint distribution and interdependencies among instructions for data selection in domain-specific instruction tuning. By computing momentum-adjusted training gradients and SGD-based validation gradients, and constructing a graph with nodes as gradient representations and edges weighted by cosine similarity, it enables a gradient walk to select training samples aligned with a core knowledge extracted via PCA. The method demonstrates strong data efficiency, often matching or surpassing full-data instruction tuning with only 1% of the data, and shows robust gains on multi-task and complex reasoning tasks. This approach offers a scalable path to efficient, domain-specific LLM adaptation with reduced data requirements.

Abstract

Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning. However, the limited availability of domain-specific data significantly hampers their performance on specialized tasks. While existing methods primarily focus on selecting training data from general datasets that are similar to the target domain, they often fail to consider the joint distribution of instructions, resulting in inefficient learning and suboptimal knowledge transfer. To address these challenges, we introduce G2IS (Gradient-based Graph Instruction Selection), a novel method that constructs a mixed gradient-based instruction graph to capture the joint distribution and interdependencies between instructions. By accounting for the relationships between instructions, G2IS improves domain adaptation efficiency. Additionally, we propose a gradient walk algorithm to refine the data selection process, enhancing both training effectiveness and efficiency. Our experiments demonstrate that G2IS outperforms traditional methods across various domain adaptation tasks, yielding significant performance gains, particularly in complex, data-scarce scenarios. These results underscore the potential of G2IS in advancing the development of large, domain-specific models.

Figures (3)

• Figure 1: On the FLAN-V2 dataset, our method selects 1% of the data and compares it with the full dataset across three models. In most tasks, our approach using only 1% outperforms instruction tuning on the full dataset.
• Figure 2: The left side illustrates the core knowledge extracted from the validation set. In the center, anchor selection for the gradient walk is performed by identifying the most similar data points from the training set, based on the core knowledge. These anchors are then used to conduct a gradient walk in the graph (right side), ensuring three key conditions: No Conflict in Knowledge, Consistency with Core Knowledge, and Knowledge Coherence. Finally, a gradient subgraph is selected in the lower-right corner, representing a subset of knowledge from the training set (e.g., probabilistic knowledge).
• Figure 3: On the Infinity-Instruct dataset, we utilize the gradient walk algorithm based on principal components of knowledge extracted from different proportions of the validation set.The results of training after selecting data that enhances MMLU (upper) and GSM8K (lower) are presented.