SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training

TL;DR

SPICE addresses the gap between theory and practice in information-based data selection for instruction tuning by identifying gradient conflicts as a key driver of rapid decay in marginal information gains. It introduces an ε-decomposition of the Fisher-information objective and a conflict-aware greedy algorithm that penalizes misalignment while preserving high-information samples, plus adaptive stopping and proxy-model efficiency. Theoretical results bound greedy performance via curvature that depends on gradient inner products and ε, and empirical analyses validate the conflict-information relationship across benchmarks. Practically, SPICE achieves comparable or better downstream performance than full-data tuning while using only 10% of the data, with substantial reductions in training cost and scalable implementation via AdaFisher and proxy selection. The work highlights gradient conflicts as a central factor in data efficiency and provides a principled framework for constructing compact, high-quality instruction-tuning corpora.

Abstract

Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling greedy algorithms to achieve a $(1-1/e)$ approximation under a cardinality budget. In practice, however, we identify alleviating gradient conflicts, misalignment between per-sample gradients, is a key factor that slows down the decay of marginal log-determinant information gains, thereby preventing significant loss of information. We formalize this via an $\varepsilon$-decomposition that quantifies the deviation from ideal submodularity as a function of conflict statistics, yielding data-dependent approximation factors that tighten as conflicts diminish. Guided by this analysis, we propose SPICE, a conflict-aware selector that maximizes information while penalizing misalignment, and that supports early stopping and proxy models for efficiency. Empirically, SPICE selects subsets with higher log-determinant information than original criteria, and these informational gains translate into performance improvements: across 8 benchmarks with LLaMA2-7B and Qwen2-7B, SPICE uses only 10% of the data, yet matches or exceeds 6 methods including full-data tuning. This achieves performance improvements with substantially lower training cost.

Figures (10)

• Figure 1: (a) Concept. At step $t$, the marginal information gain $\Delta_t$ is the incremental increase of Fisher Information utility when adding one sample under the current set $S$. Slower decay of $\Delta_t$ yields larger cumulative information under the same budget $k$. (b) Empirical. A low-conflict selection (conflict measured by negative cosine alignment to the mean gradient) exhibits slower decay and thus higher information utility at equal budgets.
• Figure 2: (a) Gradient Visualization: The direction and information distribution of two representative data points in gradient 2D and 3D spaces, indicating that there are always gradient conflicts with high information in the data; (b) Decay of the high/low conflict subsets: Low conflicts can bring greater information in a limited number of steps; (c) Spearman correlation analysis: The overall conflict is negatively correlated with marginal gain $\Delta$, and positively correlated with $\varepsilon$.
• Figure 3: (a) Cost-accuracy trade-off. Average performance versus normalized time cost. SPICE/SPICE+ attain higher accuracy than Full with lower cost; SPICE+ is slightly more accurate but slower due to adaptive stopping. (b) Penalty parameter sensitivity. Varying $\lambda \in \{0,0.1,0.2,0.5,1.0\}$, performance degrades at $\lambda=0$ without penalty (non-conflict-Aware greedy selection), while $\lambda \in [0.1, 0.5]$ performs well. (c) Efficient selection. Impact of efficient selection: smaller proxy models and larger step intervals preserve performance while reducing cost.
• Figure 4: Decay of the high and low conflict subsets experiments in HumanEval, GSM8K, MMLU, Code Alpaca, Stanford Alpaca and ShareGPT. Obviously, the overall results shows highly consistent with our corollary
• Figure 5: Spearman correlation analysis experiments in HumanEval, GSM8K, MMLU, Code Alpaca, Stanford Alpaca and ShareGPT. The overall conflict is negatively correlated with marginal gain $\Delta$, and positively correlated with $\varepsilon$, which confirms our conclusion.

Theorems & Definitions (18)

• Definition 1: Fisher Information Utility
• Definition 2: Submodular Function
• Definition 3: $\mathrm{base}$ $\varepsilon$-decomposition
• Theorem 1: $\varepsilon$-submodularity
• Theorem 2: Classical greedy guarantee
• Theorem 3: Curvature-dependent guarantee
• Lemma 1: Curvature upper bound via $\varepsilon$
• Theorem 4: Perturbation bound via gradient alignment
• Corollary 1: Data-dependent approximation guarantee
• Definition 4: Gradient Conflict