COSMIR: Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context

TL;DR

COSMIR tackles the difficulty of reasoning over long contexts by replacing free-form multi-agent summaries with a centralized, structured memory and a fixed micro-cycle for workers. A Planner generates checkable sub-questions, workers perform Extract–Infer–Refine cycles that update a shared memory, and a Manager synthesizes the final answer from this memory, preserving long-range evidence and enabling auditability. In HELMET long-context QA benchmarks, COSMIR reduces information loss and improves accuracy relative to a Chain-of-Agents baseline at comparable cost, with gains tied to targeted questioning and robust memory organization. The approach augments faithfulness and long-range aggregation in reasoning tasks and offers a transparent framework adaptable to other long-context domains.

Abstract

Reasoning over very long inputs remains difficult for large language models (LLMs). Common workarounds either shrink the input via retrieval (risking missed evidence), enlarge the context window (straining selectivity), or stage multiple agents to read in pieces. In staged pipelines (e.g., Chain of Agents, CoA), free-form summaries passed between agents can discard crucial details and amplify early mistakes. We introduce COSMIR (Chain Orchestrated Structured Memory for Iterative Reasoning), a chain-style framework that replaces ad hoc messages with a structured memory. A Planner agent first turns a user query into concrete, checkable sub-questions. worker agents process chunks via a fixed micro-cycle: Extract, Infer, Refine, writing all updates to the shared memory. A Manager agent then Synthesizes the final answer directly from the memory. This preserves step-wise read-then-reason benefits while changing both the communication medium (structured memory) and the worker procedure (fixed micro-cycle), yielding higher faithfulness, better long-range aggregation, and auditability. On long-context QA from the HELMET suite, COSMIR reduces propagation-stage information loss and improves accuracy over a CoA baseline.

Figures (1)

• Figure 1: Overview of $\textsc{COSMIR}$, a training-free framework for long context tasks. It consists of a Planner agent which given the question generates clarifying sub-questions. Segmented chunks from the context are then processed by the Worker agent in a fixed micro cycle which has three phases: Extract, Infer and Refine. Through these phases the Worker agent edits a structured centralized memory by extracting facts, making logical inferences over the facts and planning next steps by removing/adding new sub-questions. Finally, the structured memory is passed to the Manager agent to generate a final coherent answer. Boxes in blue are excerpts from chunks $c_1$ and $c_R$. Key portions of these excerpts, that are needed to answer the query $q$ have been highlighted and corresponding facts that have been extracted from these chunks have been highlighted in the structured memory.