Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation

Xingyuan Pan, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, Shanbo Cheng

TL;DR

G-DIG tackles the data quality and diversity bottleneck in instruction finetuning for machine translation by proposing a gradient-based data selection framework. It combines Influence Function-based high-quality data selection with gradient-distance clustering to maximize data diversity, using a seed dataset to guide selection. Empirical results on Zh→En and De→En show robust translation gains across data budgets, with strong performance relative to baselines and competitiveness with SOTA models, plus favorable human judgments. The work also discusses computational costs and practical considerations, highlighting data-efficiency benefits and avenues for future cost reductions.

Abstract

Large Language Models (LLMs) have demonstrated remarkable abilities in general scenarios. Instruction finetuning empowers them to align with humans in various tasks. Nevertheless, the Diversity and Quality of the instruction data remain two main challenges for instruction finetuning. With regard to this, in this paper, we propose a novel gradient-based method to automatically select high-quality and diverse instruction finetuning data for machine translation. Our key innovation centers around analyzing how individual training examples influence the model during training. Specifically, we select training examples that exert beneficial influences on the model as high-quality ones by means of Influence Function plus a small high-quality seed dataset. Moreover, to enhance the diversity of the training data we maximize the variety of influences they have on the model by clustering on their gradients and resampling. Extensive experiments on WMT22 and FLORES translation tasks demonstrate the superiority of our methods, and in-depth analysis further validates their effectiveness and generalization.

G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation

TL;DR

G-DIG tackles the data quality and diversity bottleneck in instruction finetuning for machine translation by proposing a gradient-based data selection framework. It combines Influence Function-based high-quality data selection with gradient-distance clustering to maximize data diversity, using a seed dataset to guide selection. Empirical results on Zh→En and De→En show robust translation gains across data budgets, with strong performance relative to baselines and competitiveness with SOTA models, plus favorable human judgments. The work also discusses computational costs and practical considerations, highlighting data-efficiency benefits and avenues for future cost reductions.

Abstract

Large Language Models (LLMs) have demonstrated remarkable abilities in general scenarios. Instruction finetuning empowers them to align with humans in various tasks. Nevertheless, the Diversity and Quality of the instruction data remain two main challenges for instruction finetuning. With regard to this, in this paper, we propose a novel gradient-based method to automatically select high-quality and diverse instruction finetuning data for machine translation. Our key innovation centers around analyzing how individual training examples influence the model during training. Specifically, we select training examples that exert beneficial influences on the model as high-quality ones by means of Influence Function plus a small high-quality seed dataset. Moreover, to enhance the diversity of the training data we maximize the variety of influences they have on the model by clustering on their gradients and resampling. Extensive experiments on WMT22 and FLORES translation tasks demonstrate the superiority of our methods, and in-depth analysis further validates their effectiveness and generalization.
Paper Structure (29 sections, 1 theorem, 11 equations, 4 figures, 4 tables)

This paper contains 29 sections, 1 theorem, 11 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM for Machine Translation.
  4. Traning Data Quality and Diversity.
  5. Gradient-based Data Selection.
  6. Methods
  7. High-quality Data Selection
  8. Diverse Data Selection
  9. Experiment Setup
  10. Datasets.
  11. Implementation Details.
  12. Evaluation.
  13. Baselines and Comparisons.
  14. Experimental Results
  15. Main Results
  16. ...and 14 more sections

Key Result

Lemma 1

goodfellow2016deep Assuming that the model is trained via $T$ steps of Stochastic Gradient Descent (SGD) and the learning rate $\eta$ is fixed during training. Then the model parameter at the end of training $\bm{\theta}_T$ is equal to the model parameter obtained through regularized empirical risk where $\bm\Lambda$ is the diagonal matrix in the eigendecomposition of $~{\bf H}_{\bm{\theta}^*}={\

Figures (4)

  • Figure 1: Overview of our proposed method. Our overall method consists of two components: (1) high-quality data selection and (2) enhancing their diversity. In high-quality data selection, we calculate the pair-wise influence (dashed arrows) of the candidates on seed data. Then we select those with all positive influences (as marked green). Afterwards, we cluster on the selected high-quality data to distinguish dissimilar influences (as marked in dots with different colors) and resample to further obtain high-quality and diverse finetuning data.
  • Figure 2: The comparison results of model trained on various amounts of data selected by our G-DIG, G-DIG w/o Diversity and Random selection on Zh $\Rightarrow$ En and De $\Rightarrow$ En translations. We plot the results on Zh $\Rightarrow$ En and De $\Rightarrow$ En translations in the left and right two columns respectively.
  • Figure 3: The comparison results of model trained on various amounts of data selected by our G-DIG w/o Diversity, Reward model selection and Random selection on Zh $\Rightarrow$ En and De $\Rightarrow$ En translations. We plot the results on Zh $\Rightarrow$ En and De $\Rightarrow$ En translations in the left and right two columns respectively.
  • Figure 4: The comparisons between our G-DIG,G-DIG w/o Diversity and embedding-based methods on various amounts of training data on Zh $\Rightarrow$ En translation.

Theorems & Definitions (1)

  • Lemma 1