Latent Reasoning via Sentence Embedding Prediction

TL;DR

The paper demonstrates that pretrained language models can be extended from token-level generation to sentence-level reasoning by autoregressively predicting continuous sentence embeddings. It introduces two embedding paradigms—semantic (reconstruction-based) and contextual (prediction-based)—and evaluates them under discretized and continuous inference, showing that contextual embeddings with continuous inference often achieve competitive Chain-of-Thought performance while greatly reducing inference cost. The Latent Model architecture enables reasoning entirely in latent space, with SentenceLens providing human-readable interpretability of latent trajectories. Findings indicate potential scalability and modular reuse, though continuous latent reasoning remains fragile and benefits from grounding in explicit language. This work opens avenues for efficient, structured reasoning in latent spaces, with implications for multi-domain tasks and future hybrid systems that blend discrete and continuous representations.

Abstract

Autoregressive language models (LMs) generate one token at a time, yet human reasoning operates over higher-level abstractions - sentences, propositions, and concepts. This contrast raises a central question- Can LMs likewise learn to reason over structured semantic units rather than raw token sequences? In this work, we investigate whether pretrained LMs can be lifted into such abstract reasoning spaces by building on their learned representations. We present a framework that adapts a pretrained token-level LM to operate in sentence space by autoregressively predicting continuous embeddings of next sentences. We explore two embedding paradigms inspired by classical representation learning: 1) semantic embeddings, learned via autoencoding to preserve surface meaning; and 2) contextual embeddings, trained via next-sentence prediction to encode anticipatory structure. We evaluate both under two inference regimes: Discretized, which decodes each predicted embedding into text before re-encoding; and Continuous, which reasons entirely in embedding space for improved efficiency. Across four domains - mathematics, logic, commonsense, and planning - contextual embeddings under continuous inference show competitive performance with Chain-of-Thought (CoT) while reducing inference-time FLOPs on average by half. We also present early signs of scalability and modular adaptation. Finally, to visualize latent trajectories, we introduce SentenceLens, a diagnostic tool that decodes intermediate model states into interpretable sentences. Together, our results indicate that pretrained LMs can effectively transition to abstract, structured reasoning within latent embedding spaces.

Figures (6)

• Figure 1: Sentence-level reasoning framework. Training: the latent model reads the question tokens and previous embeddings, predicts $\hat{h}_t$, and a frozen decoder reconstructs $s_t$; Inference: embedding can be rolled forward by (a) Discretized: decode $\rightarrow$ text $\rightarrow$ encode or (b) Continuous: pass-through.
• Figure 2: Illustration of the different types of sentence embeddings used in our framework.
• Figure 3: Performance Change when injecting a Gaussian random noise to different modes of inferencing, for Ctx-C model in GSM8K and CSQA datasets.
• Figure 4: Evaluation Prompt used to GPT-4o for judging intermediate reasoning step's quality.
• Figure 5: Example instances from each dataset.