Latent Reasoning in LLMs as a Vocabulary-Space Superposition

TL;DR

This work tackles the computational burden of chain-of-thought reasoning in LLMs by introducing latent reasoning constrained to the vocabulary's column space. Latent-SFT uses a two-stage process: first, an encoder–decoder with LTIM/LTSuM masks learns latent tokens as linear combinations within the embedding space, and second, the model learns to autonomously generate these latent tokens while optimizing KL and CE losses, with $\mathbf{z}_i = \mathbf{E}\boldsymbol{\alpha}_i$ and $\boldsymbol{\alpha}_i = \mathrm{softmax}(\boldsymbol{\ell}_i / T)$. The approach yields competitive results with explicit CoT-SFT on GSM8k and outperforms hidden-state-based latent methods on harder datasets like Math500 and AIME24, while achieving substantial reductions in reasoning steps. Analyses show latent reasoning embodies both the compression of a single reasoning path and the superposition of multiple paths, quantified by $\mathrm{ECR@K}$, $N_{\text{eff}}$, and Top-2 scores, highlighting the capacity for parallel exploration within a constrained embedding space. Overall, Latent-SFT advances efficient reasoning with principled latent-token construction and demonstrates practical gains in both accuracy and efficiency on mathematical reasoning benchmarks.

Abstract

Large language models (LLMs) demonstrate strong reasoning abilities with chain-of-thought prompting, but explicit reasoning introduces substantial computational overhead. Recent work on latent reasoning reduces this cost by reasoning in latent space without explicit supervision, but performance drops significantly. Our preliminary experiments suggest that this degradation stems from the unstructured latent space, which makes fitting latent tokens difficult. To address this, we restrict the latent space to the column space of the LLM vocabulary, treating latent reasoning as a superposition over vocabulary probabilities. Once latent reasoning concludes, it collapses into an eigenstate of explicit reasoning to yield the final answer. Based on this idea, we propose Latent-SFT, a two-stage learning framework. In the first stage, we design two specialized attention masks to guide the Latent Token Encoder in generating latent tokens, allowing the LLM to produce the correct answer conditioned on them. In the second stage, the Latent Token Encoder is discarded, and the LLM is directly trained to generate these latent tokens autonomously for latent reasoning, optimized with KL and CE losses. Latent-SFT sets a new state of the art on GSM8k, matching explicit SFT performance while cutting reasoning chains by up to 4 times and outperforming prior latent methods. On Math500 and AIME24, lexical probability-based latent reasoning also clearly surpasses hidden-state-based approaches. Our metrics of effective compression rate and effective global parallelism further show that latent reasoning is both the compression of a single path and the superposition of multiple paths.

Figures (10)

• Figure 1: Comparison of Latent and Explicit Reasoning. Latent reasoning operates in a vocabulary superposition state, completing the reasoning process and reaching the correct answer with fewer tokens. Each step of explicit reasoning can be regarded as approximating a one-hot distribution.
• Figure 2: Visualization of last-layer hidden states and token embeddings of the LLaMA-3.2-1B-Instruct model on the GSM8k dataset. Results are shown from both global and token-specific perspectives to highlight the substantial distributional differences. In addition, we report the statistical differences between the two distributions using the Fréchet Inception Distance (FID) and the squared Maximum Mean Discrepancy (MMD$^2$). We also estimate cosine similarity via random sampling.
• Figure 3: Decay curves of singular values for embedding matrices in various LLMs. The legend indicates the effective rank of each embedding matrix.
• Figure 4: Overview of Latent-SFT. The training process consists of two phases: (a) Generating Latent Tokens via Induction–Supervision Masking. In the mask sub-figure, components such as $Q$, $S_i$ and Answer (which may span multiple tokens) are implemented via a standard autoregressive attention mechanism. The Latent Token Encoder shares both structure and initialization with the base LLM. (b) Training the LLM to Autonomously Generate Latent Tokens. In this phase, the LLM is trained to perform latent reasoning independently, using a weighted combination of KL loss and CE loss.
• Figure 5: Latent-SFT's ECR@K values on GSM8k-Aug test samples. Each curve represents the per-sample ECR@K, with samples sorted along the x-axis by their ECR@300 values.