QRMeM: Unleash the Length Limitation through Question then Reflection Memory Mechanism

TL;DR

QRMeM tackles long-context understanding by coupling a query-driven Structured Memory graph with long-term Original Segmentation, enabling targeted retrieval and reasoning. It introduces a reflection-based navigation paradigm with Entity Trial and Graph Expansion Search to iteratively discover relevant segments and learn from errors. Empirical results on MCQ (QuALITY) and Multi-doc QA benchmarks show QRMeM surpassing retrieval baselines and many long-context approaches, especially on complex multi-hop tasks. By tightly integrating structured memory with iterative, error-aware navigation, QRMeM reduces information loss and bias while maintaining efficient long-context processing.

Abstract

While large language models (LLMs) have made notable advancements in natural language processing, they continue to struggle with processing extensive text. Memory mechanism offers a flexible solution for managing long contexts, utilizing techniques such as compression, summarization, and structuring to facilitate nuanced and efficient handling of large volumes of text. However, existing techniques face challenges with static knowledge integration, leading to insufficient adaptation to task-specific needs and missing multi-segmentation relationships, which hinders the dynamic reorganization and logical combination of relevant segments during the response process. To address these issues, we introduce a novel strategy, Question then Reflection Memory Mechanism (QRMeM), incorporating a dual-structured memory pool. This pool synergizes static textual content with structured graph guidance, fostering a reflective trial-and-error approach for navigating and identifying relevant segments. Our evaluation across multiple-choice questions (MCQ) and multi-document question answering (Multi-doc QA) benchmarks showcases QRMeM enhanced performance compared to existing approaches.

Figures (5)

• Figure 1: Question then Reflection Memory mechanism is divided into two parts. The upper part, Structured Memory, is a graph constructed from entities and relations extracted from the text. The lower part, Original Segmentation, is the original segmentation. These two parts are linked by the co-occurrence relationships of entities appearing in the segments. QRMeM starts from the core entity of the question and expands on the Structured Memory to find the most relevant next entity node. It uses this to expand and revise the segment sets until the corresponding segment set can answer the question.
• Figure 2: Dual-structure Memory Construction pipeline in QRMeM, where $e$ is the entity inside each graph. ① For each segment, the sub-graph $G_k$ is initialized according to the question $Q$ to be answered, then updated by the further generated questions ${\mathbb{Q}}$ to the current graph. ② Sub-graphs are combined into global graph $G$ through entity disambiguation and relation fusion. All the segmentation ${\mathbf{S}}_k$ is linked with the entity that appeared to construct the dual-structured memory pool.
• Figure 3: Strategies for Knowledge-directed Navigation in QRMeM. Navigation starts with a subgraph containing only the target entity from the question and its linked segments. The subgraph is then expanded based on the relevance of relationship edges connected to the current entity. The process will continue until the linked segments are sufficient to answer the question.
• Figure 4: Accuracy on QuALITY datasets with different max trials number.
• Figure 5: The performance over different distances between the supporting segments.