LSTM-MAS: A Long Short-Term Memory Inspired Multi-Agent System for Long-Context Understanding

TL;DR

This work tackles long-context understanding in LLMs by introducing LSTM-MAS, a training-free multi-agent system that mimics LSTM memory through a chain of Worker, Filter, Judge, and Manager agents. Each node processes a text block and transfers an implicit state to the next, with a final Manager performing global reasoning, thereby mitigating error accumulation and hallucinations. Across eight datasets and multiple base models, LSTM-MAS substantially outperforms the state-of-the-art CoA, with notable gains such as 40.93% on NarrativeQA, 43.70% on Qasper, 121.57% on HotpotQA, and 33.12% on MuSiQue, and 16K window experiments confirm robustness beyond context length. The results support a general design principle of translating gating concepts from neural networks to multi-agent systems to enable scalable, long-context reasoning without additional training.

Abstract

Effectively processing long contexts remains a fundamental yet unsolved challenge for large language models (LLMs). Existing single-LLM-based methods primarily reduce the context window or optimize the attention mechanism, but they often encounter additional computational costs or constrained expanded context length. While multi-agent-based frameworks can mitigate these limitations, they remain susceptible to the accumulation of errors and the propagation of hallucinations. In this work, we draw inspiration from the Long Short-Term Memory (LSTM) architecture to design a Multi-Agent System called LSTM-MAS, emulating LSTM's hierarchical information flow and gated memory mechanisms for long-context understanding. Specifically, LSTM-MAS organizes agents in a chained architecture, where each node comprises a worker agent for segment-level comprehension, a filter agent for redundancy reduction, a judge agent for continuous error detection, and a manager agent for globally regulates information propagation and retention, analogous to LSTM and its input gate, forget gate, constant error carousel unit, and output gate. These novel designs enable controlled information transfer and selective long-term dependency modeling across textual segments, which can effectively avoid error accumulation and hallucination propagation. We conducted an extensive evaluation of our method. Compared with the previous best multi-agent approach, CoA, our model achieves improvements of 40.93%, 43.70%,121.57% and 33.12%, on NarrativeQA, Qasper, HotpotQA, and MuSiQue, respectively.

Figures (5)

• Figure 1: Analogical comparison between three multi-agent systems (MASs) and traditional neural network structures. Specifically, the upper left panel draws an analogy between LONGAGENT zhao2024longagent and the Jordan RNN jordan1986attractor; the lower left panel analogizes CoA zhang2024chain with the Elman RNN elman1990finding; and the right panel illustrates the analogical analysis between the proposed framework LSTM-MAS and the standard LSTM.
• Figure 2: Test Results of Qwen2.5-0.5B (left) and Qwen2.5-1.5B (right) as base models on vanilla, LightRAG, CoA, and LSTM-MAS. LSTM-MAS achieved leading results on all eight datasets.
• Figure 3: Overview of LSTM-MAS, which is organized in a chain structure. Each node includes a worker agent, a filter agent and a judge agent. At the end of the chain is a manager agent responsible for answering questions based on the output of the entire chain.
• Figure 4: Needle-in-Haystack Test Results with Qwen2.5(left) and ERNIE 3.5(right) models as Filter. This indicates that the ERNIE series models are more suitable for the role of Filter agent and also highlights that heterogeneous models may cooperate effectively. Although this test could continue indefinitely, testing was stopped when the input length reached 1500K due to time constraints.
• Figure 5: An case study of CoA(left) and LSTM-MAS(right). This demonstrates that the filter and judger in LSTM-MAS can effectively recover errors in the chain and improve the accuracy of the results.