Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Think Anywhere in Code Generation

Xue Jiang, Tianyu Zhang, Ge Li, Mengyang Liu, Taozhi Chen, Zhenhua Xu, Binhua Li, Wenpin Jiao, Zhi Jin, Yongbin Li, Yihong Dong

Abstract

Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training, then leveraging outcome-based RL rewards to drive the model's autonomous exploration of when and where to invoke reasoning. Extensive experiments on four mainstream code generation benchmarks (i.e., LeetCode, LiveCodeBench, HumanEval, and MBPP) show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches, while demonstrating consistent generalization across diverse LLMs. Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.

Think Anywhere in Code Generation

Abstract

Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training, then leveraging outcome-based RL rewards to drive the model's autonomous exploration of when and where to invoke reasoning. Extensive experiments on four mainstream code generation benchmarks (i.e., LeetCode, LiveCodeBench, HumanEval, and MBPP) show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches, while demonstrating consistent generalization across diverse LLMs. Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.

Paper Structure

This paper contains 32 sections, 10 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reasoning and Planning Mechanisms in LLMs.
  4. Post-Training of LLMs for Code Generation.
  5. Methodology
  6. Defining Think-Anywhere
  7. Upfront Thinking.
  8. Think-Anywhere.
  9. Training Template.
  10. Cold Start for Think-Anywhere
  11. Automatic Data Construction.
  12. Dedicated Reasoning Trigger Token.
  13. RLVR for Think-Anywhere
  14. Reinforcement Learning Algorithm.
  15. Reward Modeling.
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Illustration of Think-Anywhere. Reasoning can be invoked at any token position during code generation. The ellipsis ("...") within <think> or <thinkanywhere> represents truncated thinking content for brevity.
  • Figure 2: Results of Thinking Position Analysis.
  • Figure 3: Token cost of different methods.
  • Figure 4: Pass@k comparison between GRPO and Think-Anywhere across four benchmarks.