Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Speed is Confidence

Joshua V. Dillon

TL;DR

This work proposes a halting-based ensemble paradigm inspired by speed-accuracy principles observed in biological systems, where inference is driven by the first confident signal. By applying halt-first selection to ensembles of Tiny Recursive Models (TRMs) and introducing Oracle-First Training with multiple latent initializations and winner-take-all updates, the authors achieve near-ensemble performance with a single model and dramatically lower inference cost on Sudoku-Extreme. They provide supporting theory in terms of bilevel fixed points and Schur-complement gradients to connect halting speed with conditioning, and introduce practical accelerators (Muon optimizer, selective regularization, SwiGLU modification, and SVD-aligned initialization) to enable fast training. Comprehensive experiments show that halting speed is a strong proxy for confidence, with rapid early convergence, and demonstrate the method’s efficiency and robustness, albeit with Sudoku-domain limitations and a clear path for broader validation. The work suggests that speed-aware training can internalize ensemble benefits, offering a practical route to efficient, confident AI reasoning in constrained compute environments.

Abstract

Biological neural systems must be fast but are energy-constrained. Evolution's solution: act on the first signal. Winner-take-all circuits and time-to-first-spike coding implicitly treat when a neuron fires as an expression of confidence. We apply this principle to ensembles of Tiny Recursive Models (TRM). By basing the ensemble prediction solely on the first to halt rather than averaging predictions, we achieve 97.2% puzzle accuracy on Sudoku-Extreme while using 10x less compute than test-time augmentation (the baseline achieves 86.1% single-pass, 97.3% with TTA). Inference speed is an implicit indication of confidence. But can this capability be manifested as a training-only cost? Evidently yes: by maintaining K = 4 parallel latent states during training but backpropping only through the lowest-loss "winner," a single model achieves 96.9% +/- 0.6% puzzle accuracy with a single forward pass-matching TTA performance without any test-time augmentation. As in nature, this work was also resource constrained: all experimentation used a single RTX 5090. This necessitated efficiency and compelled our invention of a modified SwiGLU which made Muon viable. With Muon and K = 1 training, we exceed TRM baseline performance in 7k steps (40 min). Higher accuracy requires 36k steps: 1.5 hours for K = 1, 6 hours for K = 4.

Speed is Confidence

TL;DR

This work proposes a halting-based ensemble paradigm inspired by speed-accuracy principles observed in biological systems, where inference is driven by the first confident signal. By applying halt-first selection to ensembles of Tiny Recursive Models (TRMs) and introducing Oracle-First Training with multiple latent initializations and winner-take-all updates, the authors achieve near-ensemble performance with a single model and dramatically lower inference cost on Sudoku-Extreme. They provide supporting theory in terms of bilevel fixed points and Schur-complement gradients to connect halting speed with conditioning, and introduce practical accelerators (Muon optimizer, selective regularization, SwiGLU modification, and SVD-aligned initialization) to enable fast training. Comprehensive experiments show that halting speed is a strong proxy for confidence, with rapid early convergence, and demonstrate the method’s efficiency and robustness, albeit with Sudoku-domain limitations and a clear path for broader validation. The work suggests that speed-aware training can internalize ensemble benefits, offering a practical route to efficient, confident AI reasoning in constrained compute environments.

Abstract

Biological neural systems must be fast but are energy-constrained. Evolution's solution: act on the first signal. Winner-take-all circuits and time-to-first-spike coding implicitly treat when a neuron fires as an expression of confidence. We apply this principle to ensembles of Tiny Recursive Models (TRM). By basing the ensemble prediction solely on the first to halt rather than averaging predictions, we achieve 97.2% puzzle accuracy on Sudoku-Extreme while using 10x less compute than test-time augmentation (the baseline achieves 86.1% single-pass, 97.3% with TTA). Inference speed is an implicit indication of confidence. But can this capability be manifested as a training-only cost? Evidently yes: by maintaining K = 4 parallel latent states during training but backpropping only through the lowest-loss "winner," a single model achieves 96.9% +/- 0.6% puzzle accuracy with a single forward pass-matching TTA performance without any test-time augmentation. As in nature, this work was also resource constrained: all experimentation used a single RTX 5090. This necessitated efficiency and compelled our invention of a modified SwiGLU which made Muon viable. With Muon and K = 1 training, we exceed TRM baseline performance in 7k steps (40 min). Higher accuracy requires 36k steps: 1.5 hours for K = 1, 6 hours for K = 4.
Paper Structure (94 sections, 3 equations, 4 figures, 3 tables, 2 algorithms)

This paper contains 94 sections, 3 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Background
  4. Tiny Recursive Models
  5. Adaptive Computation Time
  6. Method
  7. The Ensemble Puzzle
  8. Halt-First Ensemble
  9. Oracle-First Training
  10. Training dynamics.
  11. Chain convergence and expunging.
  12. Counter-intuitive design.
  13. Faster Training
  14. Muon Optimizer
  15. Selective regularization.
  16. ...and 79 more sections

Figures (4)

  • Figure 1: Learning curves across 4 seeds (x182{a,b,c,d}). Left: Cell accuracy converges to 98.4--99.0%. Right: Puzzle accuracy reaches 96.2--97.6% by 36k steps. Shaded region shows $\pm$1 std. Dashed lines show baseline (86.1%) and TTA-8 (97.3%).
  • Figure 2: Halting distribution for halt-first ensemble (12 chains). 86.3% of puzzles have at least one chain halt at step 0; 94.9% within step 1.
  • Figure 3: $z_L$ convergence across heads during training. Pairwise cosine similarity exceeds 0.95 by step 4; expunging recovers compute for new samples.
  • Figure 4: Halting step correlates with puzzle difficulty (measured by backtracking solver steps). Pearson $r = 0.73$. The model takes longer on objectively harder puzzles.