Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controllable Mathematical Reasoning via Self-Optimizing Thought Vectors

Xuying LI

TL;DR

The paper addresses the lack of fine-grained controllability over internal reasoning in mathematical problem solving by introducing self-optimizing thought vectors whose selections are guided by entropy-based rewards $R = -\mathcal{H}(p)$. It presents an eight-vector architecture with a three-dimensional control framework (Depth, Length, Path) and demonstrates that entropy-driven optimization can produce focused, explainable reasoning patterns without external annotations, achieving GSM8K accuracy of 90.1% and a controllability score of 0.42 on Gemma-2-9B with LoRA. Empirical analyses reveal meaningful clustering of thought vectors and low-entropy distributions under control, along with robust controllability metrics and insightful case studies. The work suggests that internal reasoning can be steered through self-supervised entropy optimization, offering a path toward more transparent and adaptable AI reasoning across domains.

Abstract

We present a novel approach for controllable mathematical reasoning that leverages self-optimizing thought vectors with entropy minimization. Our method introduces learnable thought vectors that dynamically modulate the internal reasoning process of large language models. Using Gemma-2-9B on GSM8K, we achieve 90.1% accuracy with a controllability score of 0.42, demonstrating that entropy-based rewards effectively guide focused reasoning patterns without requiring external reward annotations. Our analysis reveals distinct thought vector clusters and consistent low-entropy distributions across control conditions, validating our framework for controllable AI reasoning.

Controllable Mathematical Reasoning via Self-Optimizing Thought Vectors

TL;DR

The paper addresses the lack of fine-grained controllability over internal reasoning in mathematical problem solving by introducing self-optimizing thought vectors whose selections are guided by entropy-based rewards . It presents an eight-vector architecture with a three-dimensional control framework (Depth, Length, Path) and demonstrates that entropy-driven optimization can produce focused, explainable reasoning patterns without external annotations, achieving GSM8K accuracy of 90.1% and a controllability score of 0.42 on Gemma-2-9B with LoRA. Empirical analyses reveal meaningful clustering of thought vectors and low-entropy distributions under control, along with robust controllability metrics and insightful case studies. The work suggests that internal reasoning can be steered through self-supervised entropy optimization, offering a path toward more transparent and adaptable AI reasoning across domains.

Abstract

We present a novel approach for controllable mathematical reasoning that leverages self-optimizing thought vectors with entropy minimization. Our method introduces learnable thought vectors that dynamically modulate the internal reasoning process of large language models. Using Gemma-2-9B on GSM8K, we achieve 90.1% accuracy with a controllability score of 0.42, demonstrating that entropy-based rewards effectively guide focused reasoning patterns without requiring external reward annotations. Our analysis reveals distinct thought vector clusters and consistent low-entropy distributions across control conditions, validating our framework for controllable AI reasoning.
Paper Structure (27 sections, 2 figures, 4 tables)

This paper contains 27 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Thought Vector Architecture
  5. Control Framework
  6. Entropy-Based Self-Optimization
  7. Experiments
  8. Setup
  9. Main Results
  10. Thought Vector Analysis
  11. Controllability Metrics
  12. Case Studies
  13. Analysis
  14. Conclusion
  15. Additional Experimental Results
  16. ...and 12 more sections

Figures (2)

  • Figure 1: Left: PCA visualization of thought vectors colored by control signals. Different control configurations produce well-separated clusters in the reduced dimensional space, with depth control showing the clearest separation. The distinct clustering validates that our entropy-based training successfully induces specialized thought vectors for different reasoning complexities without explicit supervision. Right: Controllability analysis heatmap revealing differential model responses across control dimensions. Depth control achieves the strongest adherence (81.3% match rate), successfully modulating reasoning complexity from simple arithmetic to multi-step derivations. Path control shows moderate success (41.2%), effectively switching between direct and explanatory modes. Length control remains challenging (2.7%), suggesting output verbosity requires different mechanisms than internal reasoning control.
  • Figure 2: Left: Evolution of entropy-based rewards during training, showing progression from unfocused (high entropy) to focused (low entropy) thinking. Right: Violin plots revealing bimodal thought vector activation patterns.