ReasonCACHE: Teaching LLMs To Reason Without Weight Updates

TL;DR

ReasonCache introduces a prefix-tuning based KV-cache that distills in-context demonstrations into a fixed, trainable memory while freezing the backbone model. It demonstrates that reasoning capabilities can be acquired without weight updates, achieving competitive or superior performance to in-weight methods on challenging benchmarks such as GSM8K, MATH, GPQA-Diamond, and OpenThoughts-3 subsets, with improved data, compute, and parameter efficiency. The authors formalize the expressivity advantage of KV-prefixes over low-rank adaptations and provide empirical geometry analyses showing prefixes inject novel directions beyond the base model’s value subspace. This work positions ReasonCache as a scalable middle path between in-context and in-weight learning, enabling reasoning beyond context window without parameter updates and with practical deployment benefits.

Abstract

Can Large language models (LLMs) learn to reason without any weight update and only through in-context learning (ICL)? ICL is strikingly sample-efficient, often learning from only a handful of demonstrations, but complex reasoning tasks typically demand many training examples to learn from. However, naively scaling ICL by adding more demonstrations breaks down at this scale: attention costs grow quadratically, performance saturates or degrades with longer contexts, and the approach remains a shallow form of learning. Due to these limitations, practitioners predominantly rely on in-weight learning (IWL) to induce reasoning. In this work, we show that by using Prefix Tuning, LLMs can learn to reason without overloading the context window and without any weight updates. We introduce $\textbf{ReasonCACHE}$, an instantiation of this mechanism that distills demonstrations into a fixed key-value cache. Empirically, across challenging reasoning benchmarks, including GPQA-Diamond, ReasonCACHE outperforms standard ICL and matches or surpasses IWL approaches. Further, it achieves this all while being more efficient across three key axes: data, inference cost, and trainable parameters. We also theoretically prove that ReasonCACHE can be strictly more expressive than low-rank weight update since the latter ties expressivity to input rank, whereas ReasonCACHE bypasses this constraint by directly injecting key-values into the attention mechanism. Together, our findings identify ReasonCACHE as a middle path between in-context and in-weight learning, providing a scalable algorithm for learning reasoning skills beyond the context window without modifying parameters. Our project page: https://reasoncache.github.io/

Figures (17)

• Figure 1: (Left) Comparison of adaptation mechanisms. (a) In-weight learning updates some or all model parameters. (b) In-context learning (ICL) adapts behavior by conditioning on exemplar tokens without parameter updates. (c) ReasonCache (ours) learns an entire KV cache directly (trainable prefixes) at each attention layer, compressing demonstrations into a compact cached state while keeping the backbone frozen. (Right) Test accuracy (%) of in-context and in-weight adaptation approaches on GPQA-Diamond; ReasonCache dominates both in-context and in-weight baselines, also surpassing full supervised fine-tuning.
• Figure 2: Test accuracy (%) of in-context and in-weight adaptation methods across AIME 24/25, GPQA-Diamond (shown in \ref{['fig:teaser']}), GSM8K and MATH. ReasonCache consistently outperforms in-context baselines (ICL, Prompt Tuning) and exceeds LoRA while keeping pretrained weights frozen. On GPQA-Diamond, ReasonCache surpasses full supervised fine-tuning (SFT).
• Figure 3: Accuracy as a function of training examples. ReasonCache matches ICL’s data efficiency at low data and continues to scale with additional data like in-weight methods.
• Figure 4: Accuracy as a function of inference cost measured in TFLOPs. Left: Prefill cost only. Right: Total inference cost (prefill + decoding). ReasonCache consistently dominates in-context learning, achieving better accuracy at substantially lower inference compute across both metrics. $n$ and $m$ denote the number of in-context example, and the number of tokens allocated to for ReasonCache , respectively.
• Figure 5: (Left): On GPQA-Diamond, ReasonCache achieves 11% higher accuracy than SFT while generating 34% shorter responses. LoRA produces the shortest responses but at the cost of lower accuracy. (right): We hypothesize that SFT externalizes weight-encoded procedural knowledge into explicit context during generation. Although the model has internalized reasoning strategies through weight updates, it nonetheless regenerates these patterns as explicit token sequences at inference time, leading to unnecessary verbosity. Our conjecture is that in contrast, ReasonCache stores procedural knowledge directly in the KV cache, eliminating the need for explicit externalization.

Theorems & Definitions (18)

• Proposition 1
• Proposition 2
• Remark 1
• Theorem 1
• Corollary 1: PT Strictly More Expressive than QK-LoRA
• Theorem 2
• Proposition 2
• proof
• Proposition 2
• proof