Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reuse, Don't Recompute: Efficient Large Reasoning Model Inference via Memory Orchestration

Daivik Patel, Shrenik Patel

TL;DR

The paper tackles the high cost of test-time reasoning in large reasoning models by introducing ENGRAM-R, an inference-time memory layer that outside the model weights uses typed episodic, semantic, and procedural memory to retrieve compact, auditable evidence. Retrieved records are rendered as compact Fact Cards and used with explicit citation constraints, bounding reasoning length and enabling traceable justification. On LoCoMo and LongMemEval$_ ext{S}$, ENGRAM-R delivers order-of-magnitude reductions in input and substantial reductions in reasoning tokens while maintaining or improving accuracy, especially for multi-hop and temporal reasoning. The approach demonstrates practical compute savings with real-world deployments and generalizes to domains like healthcare (HealthBench), highlighting memory reuse as a viable lever for efficient reasoning under tight budgets.

Abstract

Large reasoning models (LRMs) achieve strong accuracy through test-time scaling, generating longer chains of thought or sampling multiple solutions, but at steep costs in tokens and latency. We argue that memory is a core ingredient for efficient reasoning: when evidence already exists, models should think less by reusing structured memory instead of recomputing derivations. We present ENGRAM-R, an inference-time memory layer that integrates typed retrieval with compact fact card representations and explicit citation control. On the LoCoMo benchmark, ENGRAM-R reduces input tokens by 85% and reasoning tokens by 75% compared to full context while maintaining high accuracy. On a multi-hop slice of the LongMemEval benchmark, it achieves similar efficiency with substantial accuracy gains. These results show that memory is not only critical for long-horizon correctness but also a practical lever for efficient reasoning under tight compute, memory, and latency budgets.

Reuse, Don't Recompute: Efficient Large Reasoning Model Inference via Memory Orchestration

TL;DR

The paper tackles the high cost of test-time reasoning in large reasoning models by introducing ENGRAM-R, an inference-time memory layer that outside the model weights uses typed episodic, semantic, and procedural memory to retrieve compact, auditable evidence. Retrieved records are rendered as compact Fact Cards and used with explicit citation constraints, bounding reasoning length and enabling traceable justification. On LoCoMo and LongMemEval, ENGRAM-R delivers order-of-magnitude reductions in input and substantial reductions in reasoning tokens while maintaining or improving accuracy, especially for multi-hop and temporal reasoning. The approach demonstrates practical compute savings with real-world deployments and generalizes to domains like healthcare (HealthBench), highlighting memory reuse as a viable lever for efficient reasoning under tight budgets.

Abstract

Large reasoning models (LRMs) achieve strong accuracy through test-time scaling, generating longer chains of thought or sampling multiple solutions, but at steep costs in tokens and latency. We argue that memory is a core ingredient for efficient reasoning: when evidence already exists, models should think less by reusing structured memory instead of recomputing derivations. We present ENGRAM-R, an inference-time memory layer that integrates typed retrieval with compact fact card representations and explicit citation control. On the LoCoMo benchmark, ENGRAM-R reduces input tokens by 85% and reasoning tokens by 75% compared to full context while maintaining high accuracy. On a multi-hop slice of the LongMemEval benchmark, it achieves similar efficiency with substantial accuracy gains. These results show that memory is not only critical for long-horizon correctness but also a practical lever for efficient reasoning under tight compute, memory, and latency budgets.

Paper Structure

This paper contains 48 sections, 14 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The ENGRAM-R Architecture
  4. Problem Setup
  5. Extensions to ENGRAM
  6. 1. Aggregation $\rightarrow$ Fact Card Rendering.
  7. 2. Prompt Construction $\rightarrow$ Controlled Citation.
  8. Overview.
  9. Evaluation
  10. Benchmarks
  11. Backbone and baselines
  12. Metrics
  13. Results
  14. LoCoMo: Category-level Analysis
  15. LongMemEval$_\text{S}$: Performance Analysis
  16. ...and 33 more sections

Figures (1)

  • Figure 1: System overview of ENGRAM-R. Dialogue turns are routed into typed stores (episodic, semantic, procedural), embedded, and retrieved at query time. Retrieved items are re-rendered into compact Fact Cards, which provide atomic, anchored claims for direct citation. This design bounds input size, shortens reasoning, and enables LRMs to achieve efficient long-horizon inference. Numbers (1)–(5) mark the main components and are referenced below.