Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks

TL;DR

The paper interrogates the assumption that instruction tuning universally enhances reasoning by directly comparing base and instruction-tuned LLMs across diverse math and domain-shift benchmarks. Using $Pass@20$ on GSM8K, Math-500, Math-Perturb Hard, and MedCalc, it shows base models often outperform instruction-tuned models in zero-shot and under distribution shift, with gains from instruction tuning being unstable and highly sensitive to prompts and evaluators. It reveals that few-shot prompting helps both types of models, but base models rely less on exemplars when zero-shot performance is strong. The findings urge cautious interpretation of instruction-tuning gains, highlight evaluator-dependent biases in latex-heavy benchmarks, and advocate for robust, multi-evaluator assessments to accurately measure genuine reasoning abilities and robustness to distribution shifts.

Abstract

Instruction finetuning is standard practice for improving LLM performance, yet it remains unclear whether it enhances reasoning or merely induces surface-level pattern matching. We investigate this by evaluating base and instruction-tuned models on standard math benchmarks, structurally perturbed variants, and domain-shifted tasks. Our analysis highlights two key (often overlooked) limitations of instruction tuning. First, the performance advantage is unstable and depends heavily on evaluation settings. In zero-shot CoT settings on GSM8K, base models consistently outperform instruction-tuned variants, with drops as high as 32.67\% (Llama3-70B). Instruction-tuned models only match or exceed this performance when provided with few-shot exemplars, suggesting a reliance on specific prompting patterns rather than intrinsic reasoning. Second, tuning gains are brittle under distribution shift. Our results show that base models surpass instruction-tuned variants on the domain-specific MedCalc benchmark. Additionally, instruction-tuned models show sharp declines on perturbed datasets, indicating sensitivity to prompt structure over robust reasoning.

Figures (7)

• Figure 1: Pass@20 on Math-500. Gap between instruction and base models reducing with parameter scale.
• Figure 2: Pass@20 on GSM8K. Instruction tuning yields marginal gains here, with mid-to-large base models often surpassing their instruction-tuned counterparts.
• Figure 3: Pass@20 on GSM8K (0-shot CoT). Base models significantly outperform instruction-tuned models
• Figure S1: Manually verified examples for Math-500 test set, showing how inconsistent both the grader and math verify are.
• Figure S2: Base models versus Instruct models on Math-500 benchmark.