RAM: Towards an Ever-Improving Memory System by Learning from Communications

TL;DR

RAM tackles the immutability of pre-trained LLMs by introducing an ever-improving memory that learns from human communication. It combines a Recursive Reasoning-based Retrieval ($R^3$) with a memory-reflection module to dynamically update memory using ground truth and feedback, enabling continual knowledge growth without re-training. Across FreshQA and MQuAKE-T, RAM achieves substantial improvements in GPT4_score compared with self-knowledge and RAG-only baselines, particularly on false-premise and multi-hop questions, and demonstrates robustness through ablations and real-user studies. The work highlights RAM’s potential for lifelong learning and dynamic knowledge acquisition, while acknowledging limitations in memory capacity and outdated information, and points to future work in richer memory structures and broader task domains.

Abstract

We introduce an innovative RAG-based framework with an ever-improving memory. Inspired by humans'pedagogical process, RAM utilizes recursively reasoning-based retrieval and experience reflections to continually update the memory and learn from users' communicative feedback, namely communicative learning. Extensive experiments with both simulated and real users demonstrate significant improvements over traditional RAG and self-knowledge methods, particularly excelling in handling false premise and multi-hop questions. Furthermore, RAM exhibits promising adaptability to various feedback and retrieval methods, showcasing its potential for advancing AI capabilities in dynamic knowledge acquisition and lifelong learning.

Figures (5)

• Figure 1: Learning framework of RAM. Blue boxes indicate LLMs' in-context reasoning and the green box indicates feedback from external users. ➊ Given a new question, LLMs take multi-step reasoning and inference through self-reflection. If the current inference is the same as in previous trials, the human will provide additional hints as feedback to help LLMs better answer. ➋ Relevant knowledge is recursively retrieved from memory based on LLMs' reasoning. ➌ LLMs generate a reflected memory learning from the feedback and the ground truth to update the memory. All prompts are shortened for simplicity; refer to Appendix \ref{['appendix:prompts']} for complete templates.
• Figure 2: Evaluation on multi-hop questions using RAM.
• Figure 3: Evaluation of multi-step trials. Each sub-figure indicates step-wise average text similarity between the $Inf$ and $G$.
• Figure 4: User feedback without explanation
• Figure 5: User feedback with hints