Efficient Inference for Large Reasoning Models: A Survey

TL;DR

This survey identifies token inefficiency as a bottleneck in Large Reasoning Models and categorizes efficient inference methods into explicit compact CoT and implicit latent CoT. It comprehensively reviews CoT compression, fine-tuning on compact reasoning, and reward-based brevity, alongside latent-reasoning approaches such as distillation and latent embeddings, then evaluates them across diverse reasoning benchmarks and objective functions. The study highlights open challenges including user-driven control, interpretability-safety trade-offs, and broader domain applicability, offering practical insights and directions for future work. It also discusses promising avenues like model merging and agent routing to synergize speed and reasoning quality for real-world deployment.

Abstract

Large Reasoning Models (LRMs) significantly improve the reasoning ability of Large Language Models (LLMs) by learning to reason, exhibiting promising performance in solving complex tasks. However, their deliberative reasoning process leads to inefficiencies in token usage, memory consumption, and inference time. Thus, this survey provides a review of efficient inference methods designed specifically for LRMs, focusing on mitigating token inefficiency while preserving the reasoning quality. The overview structure of this paper is shown in Figure~\ref{fig:paper_structure}. First, we introduce a taxonomy to group the recent methods into two main categories: (a) explicit compact Chain-of-Thought (CoT), which reduces tokens while keeping the explicit reasoning structure, and (b) implicit latent CoT, which encodes reasoning steps within hidden representations instead of explicit tokens. Meanwhile, we discuss their strengths and weaknesses. Then, we conduct empirical analyses on existing methods from reasoning scenarios, object functions, and performance \& efficiency aspects. Besides, we present open challenges in this field, including human-centric controllable reasoning, trade-off between interpretability and efficiency of reasoning, ensuring the safety of efficient reasoning, and broader applications of efficient reasoning. In addition, we highlight key insights for enhancing LRMs' inference efficiency via techniques such as model merging, new architectures, and agent routers. We hope this work serves as a valuable guide, helping researchers overcome challenges in this vibrant field. A collection of efficient reasoning methods for LRMs (papers and codes) is provided at this link: https://github.com/yueliu1999/Awesome-Efficient-Inference-for-LRMs.

Figures (8)

• Figure 1: Overview of this Survey. It mainly consists of four parts: taxonomy, empirical analyses, limitations & challenges, & further improvement.
• Figure 2: Chronological Milestones of Efficient Inference for Large Reasoning Models. The time range is mainly from July 2024 to July 2025.
• Figure 3: Taxonomy of Efficient Inference for Large Reasoning Models. The large reasoning model typically outputs long CoT (left sub-figure). The recent efficient inference methods for large reasoning models are mainly classify into (a) explicit compact CoT and (b) implicit latent CoT.
• Figure 4: Flowchart of CoT Compression Methods. Each column represents one distinct kind of approach for compressing the CoT reasoning process, highlighting the key steps of each method.
• Figure 5: Flowchart of Fine-Tuning on Compact Reasoning Chains. Each column represents one kind of strategy of SFT for token efficiency.