Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CORE: Collaborative Reasoning via Cross Teaching

Kshitij Mishra, Mirat Aubakirov, Martin Takac, Nils Lukas, Salem Lahlou

TL;DR

CoRe presents a training-time collaboration framework that turns peer success into corrective signals through a two-round protocol: a cold round of independent sampling followed by a contexted rescue round where failed models receive hints from a successful peer. The reward design combines exploitation (correctness), exploration (DPP-lite diversity), and a rescue bonus, with optional cross-model complementarity terms and per-problem normalization, to maximize team success. Across GSM8K, MATH, AIME, and GPQA under a low-data regime (≤1000 examples), CoRe yields substantial gains in both single-model Pass@K and Team Pass@K without increasing model size, primarily by reducing correlated failures and promoting complementary reasoning. These results suggest training-time collaboration is a practical path to robust reasoning for small, open models in educational, scientific, and safety-critical applications, and point to future work on scalable multi-model routing and verification.

Abstract

Large language models exhibit complementary reasoning errors: on the same instance, one model may succeed with a particular decomposition while another fails. We propose Collaborative Reasoning (CORE), a training-time collaboration framework that converts peer success into a learning signal via a cross-teaching protocol. Each problem is solved in two stages: a cold round of independent sampling, followed by a contexted rescue round in which models that failed receive hint extracted from a successful peer. CORE optimizes a combined reward that balances (i) correctness, (ii) a lightweight DPP-inspired diversity term to reduce error overlap, and (iii) an explicit rescue bonus for successful recovery. We evaluate CORE across four standard reasoning datasets GSM8K, MATH, AIME, and GPQA. With only 1,000 training examples, a pair of small open source models (3B+4B) reaches Pass@2 of 99.54% on GSM8K and 92.08% on MATH, compared to 82.50% and 74.82% for single-model training. On harder datasets, the 3B+4B pair reaches Pass@2 of 77.34% on GPQA (trained on 348 examples) and 79.65% on AIME (trained on 792 examples), using a training-time budget of at most 1536 context tokens and 3072 generated tokens. Overall, these results show that training-time collaboration can reliably convert model complementarity into large gains without scaling model size.

CORE: Collaborative Reasoning via Cross Teaching

TL;DR

CoRe presents a training-time collaboration framework that turns peer success into corrective signals through a two-round protocol: a cold round of independent sampling followed by a contexted rescue round where failed models receive hints from a successful peer. The reward design combines exploitation (correctness), exploration (DPP-lite diversity), and a rescue bonus, with optional cross-model complementarity terms and per-problem normalization, to maximize team success. Across GSM8K, MATH, AIME, and GPQA under a low-data regime (≤1000 examples), CoRe yields substantial gains in both single-model Pass@K and Team Pass@K without increasing model size, primarily by reducing correlated failures and promoting complementary reasoning. These results suggest training-time collaboration is a practical path to robust reasoning for small, open models in educational, scientific, and safety-critical applications, and point to future work on scalable multi-model routing and verification.

Abstract

Large language models exhibit complementary reasoning errors: on the same instance, one model may succeed with a particular decomposition while another fails. We propose Collaborative Reasoning (CORE), a training-time collaboration framework that converts peer success into a learning signal via a cross-teaching protocol. Each problem is solved in two stages: a cold round of independent sampling, followed by a contexted rescue round in which models that failed receive hint extracted from a successful peer. CORE optimizes a combined reward that balances (i) correctness, (ii) a lightweight DPP-inspired diversity term to reduce error overlap, and (iii) an explicit rescue bonus for successful recovery. We evaluate CORE across four standard reasoning datasets GSM8K, MATH, AIME, and GPQA. With only 1,000 training examples, a pair of small open source models (3B+4B) reaches Pass@2 of 99.54% on GSM8K and 92.08% on MATH, compared to 82.50% and 74.82% for single-model training. On harder datasets, the 3B+4B pair reaches Pass@2 of 77.34% on GPQA (trained on 348 examples) and 79.65% on AIME (trained on 792 examples), using a training-time budget of at most 1536 context tokens and 3072 generated tokens. Overall, these results show that training-time collaboration can reliably convert model complementarity into large gains without scaling model size.
Paper Structure (47 sections, 2 theorems, 22 equations, 8 figures, 7 tables, 1 algorithm)

This paper contains 47 sections, 2 theorems, 22 equations, 8 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reasoning via prompting, search, and inference-time ensembling.
  4. Reinforcement learning and preference optimization for LLM post-training.
  5. Diversity, exploration, and reducing correlated failures.
  6. Multi-agent and multi-model collaboration.
  7. Collaborative learning, distillation, and self-training with rationales.
  8. Problem Formulation
  9. Team success.
  10. Method
  11. Round A (cold).
  12. Teacher context construction.
  13. Round B (contexted rescue).
  14. Reward Design
  15. Exploitation reward.
  16. ...and 32 more sections

Key Result

Theorem 4.1

Let $C_1,C_2\in\{0,1\}$ indicate whether models 1 and 2 solve an instance, and let $T=\mathbb{I}[C_1\lor C_2]$ denote the team-success indicator. Define accuracies $p_i=\mathbb{P}(C_i=1)$ and joint success $p_{12}=\mathbb{P}(C_1=1,C_2=1)$. Then Assume without loss of generality that $p_2\ge p_1$. Then the collaboration gain over the best single model, $\Delta \triangleq \mathbb{P}(T=1)-p_2$, equa

Figures (8)

  • Figure 1: CoRe: Collaborative Reasoning via Cross Teaching Training.
  • Figure 2: CoRe training. In Round A, each model samples $K$ independent traces. If any trace succeeds, the best one is used as teacher context for Round B, where models resample with explore--exploit rewards and a rescue bonus for successful recoveries.
  • Figure 3: Comparison for Base/SD-E$^2$ oracle baselines vs. CoRe Team with Qwen-2.5-3b-instrcut and Qwen-3-4b-instrcut
  • Figure 4: Comparison for Base/SD-E$^2$ oracle baselines vs. CoRe Team with Phi-4-mini-reasoning and ministral3-3b-reasoning
  • Figure 5: Train curves for AIME with Phi-4-mini-reasoning and ministral3-3b-reasoning
  • ...and 3 more figures

Theorems & Definitions (5)

  • Theorem 4.1: Collaboration gain equals complementary correctness
  • proof
  • Remark 4.2: Limits of collaboration gain
  • Proposition 2.1: Separation of the diverse set
  • proof