Towards Efficient Medical Reasoning with Minimal Fine-Tuning Data

TL;DR

This work tackles inefficiency in SFT for medical reasoning by introducing DIQ, a data-selection framework that jointly models sample difficulty and optimization influence. Difficulty is estimated by a BiomedBERT classifier, while influence is approximated via a gradient-dot-product with a small validation set, enabling a light-weight, Hessian-free ranking of samples. By selecting data in priority quadrants of high difficulty and high influence, DIQ enables full or near-full performance with only $1\%$ of data and consistently outperforms baselines at $10\%$, with additional evidence from QA tasks and clinician-aligned reasoning signals. The approach shows notable data efficiency, reduced computational cost, and generalization across model scales and preference learning, highlighting its practical impact for deploying medical reasoning systems with constrained resources.

Abstract

Supervised Fine-Tuning (SFT) plays a pivotal role in adapting Large Language Models (LLMs) to specialized domains such as medical reasoning. However, existing SFT practices often rely on unfiltered datasets that contain redundant and low-quality samples, leading to substantial computational costs and suboptimal performance. Although existing methods attempt to alleviate this problem by selecting data based on sample difficulty, defined by knowledge and reasoning complexity, they overlook each sample's optimization utility reflected in its gradient. Interestingly, we find that gradient-based influence alone favors easy-to-optimize samples that cause large parameter shifts but lack deep reasoning chains, while difficulty alone selects noisy or overly complex cases that fail to guide stable optimization. Based on this observation, we propose a data selection strategy, Difficulty-Influence Quadrant (DIQ), which prioritizes samples in the high-difficulty-high-influence quadrant to balance complex clinical reasoning with substantial gradient influence, enabling efficient medical reasoning with minimal fine-tuning data. Furthermore, Human and LLM-as-a-judge evaluations show that DIQ-selected subsets demonstrate higher data quality and generate clinical reasoning that is more aligned with expert practices in differential diagnosis, safety check, and evidence citation, as DIQ emphasizes samples that foster expert-like reasoning patterns. Extensive experiments on medical reasoning benchmarks demonstrate that DIQ enables models fine-tuned on only 1% of selected data to match full-dataset performance, while using 10% consistently outperforms baseline methods, highlighting the superiority of principled data selection over brute-force scaling. The code and data are available at https://github.com/mihara-bot/DIQ.

Figures (17)

• Figure 1: (a) The FineMed dataset distributed by difficulty and influence scores, with data points colored by quadrant. (b) For each quadrant, the bar shows the intrinsic reasoning quality of the data and the resulting downstream performance of a Qwen3-8B model fine-tuned on that subset.
• Figure 2: Overview of the DIQ framework. Each sample is projected to a two-dimensional space using (i) a BiomedBERT classifier to produce a scalar difficulty score (one chosen dimension among Knowledge/Reasoning/Overall; Sec. \ref{['Sec:Difficulty']}), and (ii) an influence score, Dot, computed as the inner product between the sample gradient and the mean validation gradient (Eq. \ref{['eq:dot']}). Using a percentile threshold on difficulty and the median of Dot, the dataset is partitioned into four quadrants. DIQ selects data by priority ($\mathcal{Q}_1 \to \mathcal{Q}_2 \to \mathcal{Q}_3 \to \mathcal{Q}_4$), sorting within each quadrant by Dot (ties by difficulty), until the target retention ratio is reached.
• Figure 3: Downstream task performance comparison of models trained on MedReason-QA data selected from different methods at different data keeping ratios. Red line denotes DIQ, and blue line denotes random selection.
• Figure 4: The FLOPs consumption ($10^{14}$) comparison of computing DIQ scores, fine-tuning Llama3.1 and Qwen3 series models. The y-axis is log scale for better presentation.
• Figure 5: Downstream task performance of Qwen3-series models trained on DIQ-selected Huatuo at different influence scores.