MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning

TL;DR

This paper tackles the challenge of insufficient diversity in instruction-tuning data for LLMs by proposing MDIT, a model-free approach that performs cross-task embedding-space interpolation and diversity-based clustering to generate and curate diverse instruction data without external resources. By expanding the task distribution with embedding-level mixtures and selecting dense, representative samples via K-Means clustering, MDIT improves generalization across general QA, math reasoning, and code generation benchmarks, outperforming state-of-the-art data-selection and synthesis methods. Key findings show consistent gains across multiple LLaMA base models, with notable improvements at smaller data budgets and when carefully tuning interpolation parameters (e.g., \\alpha) and generated-sample counts (\\, T). The approach offers a labor-efficient, scalable data synthesis alternative that can enhance instruction tuning in diverse, complex environments while reducing reliance on external models and annotations.

Abstract

As Large Language Models (LLMs) are increasingly applied across various tasks, instruction tuning has emerged as a critical method for enhancing model performance. However, current data management strategies face substantial challenges in generating diverse and comprehensive data, restricting further improvements in model performance. To address this gap, we propose MDIT, a novel model-free data interpolation method for diverse instruction tuning, which generates varied and high-quality instruction data by performing task interpolation. Moreover, it contains diversity-based clustering strategies to ensure the diversity of the training data. Extensive experiments show that our method achieves superior performance in multiple benchmark tasks. The LLMs finetuned with MDIT show significant improvements in numerous tasks such as general question answering, math reasoning, and code generation. MDIT offers an efficient and automatic data synthetic method, generating diverse instruction data without depending on external resources while expanding the application potential of LLMs in complex environments.

Figures (3)

• Figure 1: The framework of our method MDIT. MDIT consists of two primary steps: Embedding-based Interpolation and Diversity-based Clustering: In the first step, we perform task interpolation within the high-dimensional embedding space, generating new tasks that capture diverse semantic relationships. The second step involves clustering filtering to the curated set and selecting diverse training data from each cluster for instruction tuning.
• Figure 2: The left figure shows the t-SNE plots of multiple datasets enhanced with MDIT. Red indicates the original data, while blue represents the newly generated data produced by MDIT. The two right figures show the performance scaling of the 1.3B & 7B model with the MDIT under different $k$ values.
• Figure 3: Performance of MDIT on Sheared-Llama-1.3B model under different generation sample number per original sample pair $T$.