Influence Functions for Efficient Data Selection in Reasoning

TL;DR

Quality, not merely quantity, of chain-of-thought data is critical for fine-tuning reasoning in LLMs. The authors extend influence functions to measure the causal impact of individual reasoning examples on downstream accuracy and propose influence-based pruning to select high-signal data. Experiments show that this approach outperforms perplexity and embedding baselines within a model family, particularly on GSM8k and OlympiadBench, though cross-model transfer remains inconsistent. The work highlights a principled, though computationally demanding, path to efficient data curation for reasoning tasks and suggests future directions to broaden applicability across models and datasets.

Abstract

Fine-tuning large language models (LLMs) on chain-of-thought (CoT) data shows that a small amount of high-quality data can outperform massive datasets. Yet, what constitutes "quality" remains ill-defined. Existing reasoning methods rely on indirect heuristics such as problem difficulty or trace length, while instruction-tuning has explored a broader range of automated selection strategies, but rarely in the context of reasoning. We propose to define reasoning data quality using influence functions, which measure the causal effect of individual CoT examples on downstream accuracy, and introduce influence-based pruning, which consistently outperforms perplexity and embedding-based baselines on math reasoning within a model family.

Figures (4)

• Figure 1: Influence scores $s_C(d), s_I(d)$ for training examples $d \in \mathcal{D}$. Points farther left contribute less to correct answers, while points higher up push the model toward wrong answers. Interestingly, the majority of outliers appear in the top-left quadrant, where lie the datapoints we predict to be most harmful. Note: as the final pruning is affected also by rank scores $r(d)$, few unpruned points may also be found in such quadrant.
• Figure 2: Histograms of the influence and rank scores measured across the training set $\mathcal{D}$.
• Figure 3: Influence scores $s_C(d), s_I(d)$ for training examples $d \in \mathcal{D}$ with aggressive 50% data pruning strategy (Combined).
• Figure 4: $\Delta$pass@1 vs. Random pruning on benchmarks.(a) For LLaMA-3-8B-Instruct, our IF-based Combined pruning achieves larger gains than baselines, particularly on GSM8k and OlympiadBench. On GPQA, Combined mitigates this degradation more effectively than other pruning strategies. (b) For Qwen2.5-Math-7B-Instruct ( 10% pruning), improvements do not transfer reliably: all methods remain close to Random, with AMC23 showing inconsistent deviations.