Learning from Peers in Reasoning Models

TL;DR

The Prefix Dominance Trap reveals that large reasoning models (LRMs) struggle to recover from short, poor reasoning prefixes, limiting self-verification. The authors propose Learning from Peers (LeaP), an inference-time mechanism that enables cross-path sharing of concise intermediate reasoning and routing of peer insights to improve reasoning, with Summarization and Routing stages and three routing strategies. They extend LeaP with LeaP-T for smaller models via supervised fine-tuning, achieving robust gains across AIME, AIMO, and GPQA benchmarks and sometimes matching much larger baselines. Extensive analyses show LeaP improves error correction, reduces aha moments, and remains robust under noisy peers and varying difficulty, supported by human verification and open-source releases. The work highlights a practical path to enhancing LRMs' reasoning through peer collaboration, with future directions in reinforcement learning and heterogeneous peer expertise.

Abstract

Large Reasoning Models (LRMs) have the ability to self-correct even when they make mistakes in their reasoning paths. However, our study reveals that when the reasoning process starts with a short but poor beginning, it becomes difficult for the model to recover. We refer to this phenomenon as the "Prefix Dominance Trap". Inspired by psychological findings that peer interaction can promote self-correction without negatively impacting already accurate individuals, we propose **Learning from Peers** (LeaP) to address this phenomenon. Specifically, every tokens, each reasoning path summarizes its intermediate reasoning and shares it with others through a routing mechanism, enabling paths to incorporate peer insights during inference. However, we observe that smaller models sometimes fail to follow summarization and reflection instructions effectively. To address this, we fine-tune them into our **LeaP-T** model series. Experiments on AIME 2024, AIME 2025, AIMO 2025, and GPQA Diamond show that LeaP provides substantial improvements. For instance, QwQ-32B with LeaP achieves nearly 5 absolute points higher than the baseline on average, and surpasses DeepSeek-R1-671B on three math benchmarks with an average gain of 3.3 points. Notably, our fine-tuned LeaP-T-7B matches the performance of DeepSeek-R1-Distill-Qwen-14B on AIME 2024. In-depth analysis reveals LeaP's robust error correction by timely peer insights, showing strong error tolerance and handling varied task difficulty. LeaP marks a milestone by enabling LRMs to collaborate during reasoning. Our code, datasets, and models are available at https://learning-from-peers.github.io/ .

Figures (32)

• Figure 1: The results of starting with bad beginnings.
• Figure 2: The illustration of (a) Independent Reasoning and (b) the proposed method Learning from Peers (LeaP). In independent reasoning, multiple paths are generated independently in parallel. In contrast, LeaP inserts a LeaP block into reasoning path, encouraging the model to learn from peers.
• Figure 3: An example of how LeaP enables communication between path $i$ and $j$. Text in red indicates the current path is incorrect. Text in green shows a correct summary received from a peer.
• Figure 4: The results of starting with bad beginnings for models with LeaP.
• Figure 5: We illustrate the average number of tokens and "Aha" moments on QwQ-32B. Our method produces a comparable number of tokens to the baseline, while yielding fewer "Aha" moments.