Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distilling Feedback into Memory-as-a-Tool

Víctor Gallego

TL;DR

The paper tackles the high cost and episodic nature of inference-time reasoning by introducing Distilling Feedback into Memory-as-a-Tool, which converts transient critiques into persistent, retrievable guidelines stored in a file-based memory and accessed via explicit tool calls. It formalizes a probabilistic framework where generated outputs, feedback, and memory updates are linked, and defines retrieval and consolidation as two core operations that enable memory-guided, zero-shot refinement. Empirically, the approach, evaluated on the Rubric Feedback Bench, achieves performance comparable to costly test-time refinement while significantly reducing inference cost, and demonstrates robust knowledge consolidation over long horizons. The work contributes a transparent, memory-based mechanism for continual improvement that generalizes across task types and offers a practical, interpretable alternative to embedding-based retrieval and continual fine-tuning.

Abstract

We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls. We evaluate this method on the Rubric Feedback Bench, a novel dataset for rubric-based learning. Experiments demonstrate that our augmented LLMs rapidly match the performance of test-time refinement pipelines while drastically reducing inference cost.

Distilling Feedback into Memory-as-a-Tool

TL;DR

The paper tackles the high cost and episodic nature of inference-time reasoning by introducing Distilling Feedback into Memory-as-a-Tool, which converts transient critiques into persistent, retrievable guidelines stored in a file-based memory and accessed via explicit tool calls. It formalizes a probabilistic framework where generated outputs, feedback, and memory updates are linked, and defines retrieval and consolidation as two core operations that enable memory-guided, zero-shot refinement. Empirically, the approach, evaluated on the Rubric Feedback Bench, achieves performance comparable to costly test-time refinement while significantly reducing inference cost, and demonstrates robust knowledge consolidation over long horizons. The work contributes a transparent, memory-based mechanism for continual improvement that generalizes across task types and offers a practical, interpretable alternative to embedding-based retrieval and continual fine-tuning.

Abstract

We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls. We evaluate this method on the Rubric Feedback Bench, a novel dataset for rubric-based learning. Experiments demonstrate that our augmented LLMs rapidly match the performance of test-time refinement pipelines while drastically reducing inference cost.
Paper Structure (38 sections, 3 figures, 4 tables)

This paper contains 38 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Framework
  3. Probabilistic formulation
  4. The Memory-as-a-Tool protocol
  5. Retrieval operation ($x \sim p(\cdot|z, M)$).
  6. Distillation and consolidation operation ($M' \sim p(\cdot|M, feed)$).
  7. Rubric Feedback Bench
  8. Experiments
  9. Continual Learning performance
  10. Long-Horizon Mixed-Task Experiment
  11. Cost-Efficiency Analysis
  12. Related Work
  13. Inference-time self-correction and reasoning.
  14. Optimization of specifications and prompts.
  15. Learning from feedback.
  16. ...and 23 more sections

Figures (3)

  • Figure 1: High-level overview of the proposed framework
  • Figure 2: Learning trajectories on Rubric Feedback Bench.
  • Figure 3: Cost-Score Pareto Frontier (Claude Sonnet 4.5)