m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models

TL;DR

This work presents m1, a lightweight, inference-time strategy that leverages test-time scaling to improve medical reasoning with minimal fine-tuning. By curating a high-quality reasoning dataset and applying supervision alongside a controllable thinking budget and Wait-token mechanism, m1 demonstrates robust gains on a broad set of medical QA benchmarks, with a 7B model achieving state-of-the-art performance among ≤10B parameters and a 32B model rivaling larger 70B+ systems. The study reveals an optimal reasoning budget around 4K tokens and shows that gains from extended reasoning saturate beyond this point, while budget forcing yields limited or negative benefits due to knowledge bottlenecks. Crucially, the results indicate that improvements in medical reasoning rely more on richer medical grounding through data quality and model capacity than on deeper reasoning alone, highlighting fundamental differences from mathematical domains. The authors also provide open-source data, models, and inference code to accelerate future research in inference-time optimization for clinical AI.

Abstract

Test-time scaling has emerged as a powerful technique for enhancing the reasoning capabilities of large language models. However, its effectiveness in medical reasoning remains uncertain, as the medical domain fundamentally differs from mathematical tasks in terms of knowledge representation and decision-making processes. In this paper, we provide the first comprehensive investigation of test-time scaling for medical reasoning and present m1, a simple yet effective approach that increases a model's medical reasoning capability at inference. Our evaluation across diverse medical tasks demonstrates that test-time scaling consistently enhances medical reasoning, enabling lightweight fine-tuned models under 10B parameters to establish new state-of-the-art performance, while our 32B model rivals previous 70B-scale medical LLMs. However, we identify an optimal reasoning token budget of approximately 4K, beyond which performance may degrade due to overthinking. Budget forcing, which extends test-time computation through iterative prompts, helps models double-check answers but does not necessarily improve the overall medical QA performance and, in some cases, even introduces errors into previously correct responses. Our case-by-case analysis identifies insufficient medical knowledge as a key bottleneck that prevents further performance gains through test-time scaling. We find that increasing data scale, improving data quality, and expanding model capacity consistently enhance medical knowledge grounding, enabling continued performance improvements, particularly on challenging medical benchmarks where smaller models reach saturation. These findings underscore fundamental differences between medical and mathematical reasoning in LLMs, highlighting that enriched medical knowledge, other than increased reasoning depth alone, is essential for realizing the benefits of test-time scaling.

Figures (7)

• Figure 1: Test-time scaling of m1 series. Each plot shows accuracy (%) vs. reasoning token budget for different m1 model variants on various medical QA datasets. All models improve steadily as the thinking length increases, with the 32B model reaching the best accuracy. The linear regression lines are marked in dot line with their 95% confidence interval.
• Figure 2: An overview of our data curation and training pipeline. We start with 196K raw medical QA examples, apply difficulty filtering (retaining 37K that Qwen2.5-7B-Instruct yang2024qwen2 or its 32B version cannot solve), then use DeepSeek-R1 guo2025deepseek to generate reasoning and keep correct solutions (m23K). We perform diversity sampling to select a 1K high-quality subset (m1K). These datasets are used to fine-tune base models (Qwen2.5 7B and 32B Instruct) via Supervised Fine-Tuning (SFT), resulting in the m1 models (m1-7B-1K, m1-7B-23K, m1-32B-1K).
• Figure 3: Force thinking for different evaluation datasets. Accuracy vs. number of budget forcing times (iterations of injecting "Wait") for each m1 model (7B-1K, 7B-23K, 32B-1K). A value of 0 means the model’s first answer is taken without forcing, while higher values mean the model was compelled to reconsider up to that many times (within a 2048-token limit).
• Figure 4: A failure case of test-time scaling with the Qwen2.5-7B using 1K reasoning data. Although the m1-7B-1K conducts the longest reasoning, its deficiency in essential medical knowledge prevents it from producing the right answer. On the other hand, both m1-7B-23K and m1-32B-1K effectively resolve the question with a relatively brief reasoning procedure.
• Figure 5: A failure case of budget forcing. Initially, the model produces the correct answer, but forcing it to re-think causes the model to retrieve incorrect knowledge, ultimately resulting in an erroneous answer.