Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons

Haonan Dong, Kehan Jiang, Haoran Ye, Wenhao Zhu, Zhaolu Kang, Guojie Song

Abstract

Large Reasoning Models (LRMs) have recently achieved remarkable success in complex reasoning tasks. However, closer scrutiny reveals persistent failure modes compromising performance and cost: I) Intra-step level, marked by calculation or derivation errors; II) Inter-step level, involving oscillation and stagnation; and III) Instance level, causing maladaptive over-thinking. Existing endeavors target isolated levels without unification, while their black-box nature and reliance on RL hinder explainability and controllability. To bridge these gaps, we conduct an in-depth white-box analysis, identifying key neurons (Mixture of Neurons, MoN) and their fluctuation patterns associated with distinct failures. Building upon these insights, we propose NeuReasoner, an explainable, controllable, and unified reasoning framework driven by MoN. Technically, NeuReasoner integrates lightweight MLPs for failure detection with a special token-triggered self-correction mechanism learned via SFT. During inference, special tokens are inserted upon failure detection to actuate controllable remedial behaviors. Extensive evaluations across six benchmarks, six backbone models (8B~70B) against nine competitive baselines, demonstrate that NeuReasoner achieves performance gains of up to 27.0% while reducing token consumption by 19.6% ~ 63.3%.

NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons

Abstract

Large Reasoning Models (LRMs) have recently achieved remarkable success in complex reasoning tasks. However, closer scrutiny reveals persistent failure modes compromising performance and cost: I) Intra-step level, marked by calculation or derivation errors; II) Inter-step level, involving oscillation and stagnation; and III) Instance level, causing maladaptive over-thinking. Existing endeavors target isolated levels without unification, while their black-box nature and reliance on RL hinder explainability and controllability. To bridge these gaps, we conduct an in-depth white-box analysis, identifying key neurons (Mixture of Neurons, MoN) and their fluctuation patterns associated with distinct failures. Building upon these insights, we propose NeuReasoner, an explainable, controllable, and unified reasoning framework driven by MoN. Technically, NeuReasoner integrates lightweight MLPs for failure detection with a special token-triggered self-correction mechanism learned via SFT. During inference, special tokens are inserted upon failure detection to actuate controllable remedial behaviors. Extensive evaluations across six benchmarks, six backbone models (8B~70B) against nine competitive baselines, demonstrate that NeuReasoner achieves performance gains of up to 27.0% while reducing token consumption by 19.6% ~ 63.3%.

Paper Structure

This paper contains 65 sections, 8 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Failure Attribution.
  3. Research Dilemma.
  4. Present Framework.
  5. Dive into Neurons
  6. MoN Identification
  7. Mixture of Neurons.
  8. Obs.❶ Distinct neuron experts contribute significantly to failure modes of different levels.
  9. Fluctuation Analysis
  10. Obs.❷ Corresponding experts within the MoN exhibit distinct fluctuation patterns across failure levels.
  11. Method
  12. Pattern Prediction
  13. Preliminary.
  14. I) Intra-step level.
  15. II) Inter-step level.
  16. ...and 50 more sections

Figures (5)

  • Figure 1: Distribution and illustration of failure modes across different levels using DeepSeek-R1-Distill-Qwen-7B on MATH math.
  • Figure 2: (Upper) Distribution of key neurons across distinct failure modes using DeepSeek-R1-Distill-Qwen-7B on MATH. (Lower) Time- and frequency-domain analysis of MoN for positive and negative sample pairs.
  • Figure 3: The overview of our proposed NeuReasoner.
  • Figure 4: Case studies of NeuReasoner.
  • Figure 5: Test-time scalability under self-consistency.