GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation

TL;DR

This work proposes GAM-RAG, a training-free framework that accumulates retrieval experience from recurring or related queries and updates retrieval memory over time, and builds a lightweight, relation-free hierarchical index whose links capture potential co-occurrence rather than fixed semantic relations.

Abstract

Retrieval-Augmented Generation (RAG) grounds large language models with external evidence, but many implementations rely on pre-built indices that remain static after construction. Related queries therefore repeat similar multi-hop traversal, increasing latency and compute. Motivated by schema-based learning in cognitive neuroscience, we propose GAM-RAG, a training-free framework that accumulates retrieval experience from recurring or related queries and updates retrieval memory over time. GAM-RAG builds a lightweight, relation-free hierarchical index whose links capture potential co-occurrence rather than fixed semantic relations. During inference, successful retrieval episodes provide sentence-level feedback, updating sentence memories so evidence useful for similar reasoning types becomes easier to activate later. To balance stability and adaptability under noisy feedback, we introduce an uncertainty-aware, Kalman-inspired gain rule that jointly updates memory states and perplexity-based uncertainty estimates. It applies fast updates for reliable novel signals and conservative refinement for stable or noisy memories. We provide a theoretical analysis of the update dynamics, and empirically show that GAM-RAG improves average performance by 3.95% over the strongest baseline and by 8.19% with 5-turn memory, while reducing inference cost by 61%. Our code and datasets are available at: https://anonymous.4open.science/r/GAM_RAG-2EF6.

Figures (6)

• Figure 1: Left: Traditional graph-based RAG relies on a static, stateless graph index; related queries repeatedly traverse similar paths, resulting in redundant reasoning and limited adaptability. Right: Memory-plastic RAG distills retrieval feedback and performs memory updates, so future retrieval becomes more efficient and better aligned with evolving query needs.
• Figure 2: Overview of our method. GAM-RAG introduce a training-free, gain-adaptive sentence memory to retrieval. A query activates entity nodes and runs iterative graph propagation to discover multi-hop evidence. After each episode, a Kalman-inspired, uncertainty-aware gain update adjusts both memory states and their perplexities to keep updates stable under noisy signals.
• Figure 3: Kalman gain dynamics: under different observation-noise levels $R_i$, illustrating fast warm-up and damped refinement.
• Figure 4: Ablation under long-term memorization. Performance trends over 10 turns on five datasets for GAM-RAG.
• Figure 5: Sentence-memory trajectories. Blue points denote queries, gray points are inactive sentences, and colored points are activated sentences; lighter-to-darker indicates earlier-to-later memorization turns. A sentence’s displacement across turns reflects the magnitude of its memory update. S1 denotes a supportive sentence, whereas S2 denotes a non-supportive sentence.