Code-enabled language models can outperform reasoning models on diverse tasks

TL;DR

CodeAdapt shows that standard instruct LMs can achieve strong, multi-step reasoning without RL-based fine-tuning when augmented with iterative code execution and light in-context learning. By uniting natural language reasoning with a Python-based execution workspace and a Generalization-guided Few-shot Learning (GFL) procedure, CodeAdapt matches or outperforms corresponding RL-trained reasoning models across eight diverse tasks, with notable improvements in token efficiency. The approach reveals robust, domain-general reasoning capabilities, supported by ablation and analysis showing diverse problem-solving strategies and metacognitive behaviors. These findings suggest a scalable, low-cost foundation for reasoning in AI systems that complements or rivals expensive RL-based methods.

Abstract

Reasoning models (RMs), language models (LMs) trained with reinforcement learning to produce long-form natural language reasoning, have been remarkably successful, but they still require large amounts of computation and data to train, and can be slow and expensive to run. In this paper, we show that standard instruct LMs can already be elicited to be strong reasoners at a level comparable to or even surpassing their corresponding RMs (e.g., DeepSeek V3 vs R1) without finetuning, across diverse domains from instruction following and creative generation to mathematical reasoning. This is achieved by CodeAdapt, our simple recipe that combines the CodeAct framework, where LMs interleave natural language reasoning with code execution in a multi-step fashion, with few-shot bootstrap in-context learning from as few as five training problems. Analyzing four matched pairs of LMs and RMs, we find that CodeAdapt enables three LMs to outperform the corresponding RMs on average over eight tasks (up to 22.9%) while being 10-81% more token efficient, and delivers superior performance on six tasks when averaged over the four models (up to 35.7%). Furthermore, the code-augmented reasoning traces display rich and varied problem-solving strategies. Our findings support that (1) CodeAdapt-style learning and reasoning may be robust and domain general and (2) code-enabled LMs are cognitively grounded and powerful systems, potentially providing a strong foundation for in-weight reinforcement learning.

Figures (6)

• Figure 1: CodeAdapt architecture and performance. (a) We combine an iterative code-execution LM reasoning framework inspired by CodeAct wang2024executable with (b) a novel lightweight in-context learning procedure, and (c) show that this simple combination can help a set of LMs from different providers outperform corresponding reasoning models evaluated on eight diverse tasks.
• Figure 2: CodeAdapts performs better with more budget. Average over all tasks $\pm$ SEM.
• Figure 3: Token usage for different DeepSeek model setups across tasks. Each violin plot represents the distribution of output tokens spent over the set of evaluation problems for each task. Dots indicate the means. For most tasks, reasoning models use more tokens than iterative code-enabled strategies like CodeAct and CodeAdapt.
• Figure 4: Pairwise cosine distance between embeddings of reasoning strategies.
• Figure 5: Token usage for different models, tasks, and methods. Each violin plot represents the distribution of output tokens spent over the set of evaluation problems for each task. Dots indicate the means. For most tasks, CodeAdapt uses significantly fewer tokens than reasoning models.