MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancement

TL;DR

MARINE tackles the limitation of single-round reasoning by proposing a multi-agent trajectory-refinement framework that uses a persistent reference trajectory. A refinement operator aggregates diverse agent proposals and updates the trajectory through conflict-aware meta-verification, yielding monotone improvements without parameter updates. The authors provide theoretical results on optimal batch-size under fixed budgets and logarithmic batching for unlimited budgets, and validate the approach on BrowserComp-ZH, achieving state-of-the-art results with a 685B model and strong parameter efficiency with an 80B model. The work demonstrates that inference-time trajectory refinement can match or exceed much larger baselines while substantially reducing parameter counts, with implications for post-training efficiency and reliability in real-world deployments.

Abstract

Large Language Model (LLM)-based agents demonstrate advanced reasoning capabilities, yet practical constraints frequently limit outputs to single responses, leaving significant performance potential unrealized. This paper introduces MARINE (Multi-Agent Recursive IN-context Enhancement), a theoretically grounded framework that reconceptualizes test-time reasoning as iterative refinement of a persistent reference trajectory, fundamentally departing from conventional one-shot or multi-sample paradigms. The MARINE refinement operator systematically converts a base model's pass@N capabilities into near-optimal pass@1 performance. Rigorous theoretical analysis establishes that minimal feasible batches maximize expected performance gains under fixed invocation budgets, while logarithmically growing batch schedules ensure continuous improvement without computational constraints. Comprehensive evaluation on the BrowserComp-ZH benchmark demonstrates state-of-the-art results, with a 685B-parameter implementation achieving 46.0% pass@1 accuracy. Meanwhile, MARINE establishes a new paradigm for parameter-efficient reasoning: an 80B-parameter model augmented with MARINE matches the performance of standalone 1000B-parameter agents, reducing parameter requirements by over an order of magnitude. Notably, within a fixed computational budget, the proposed MARINE delivers higher-quality samples to alignment and optimization processes than traditional sampling-and-ranking strategies. Consequently, it has great potential to boost post-training efficiency.

Figures (3)

• Figure 1: MARINE Framework: Multi-Agent Recursive IN-Context Enhancement for Trajectory Refinement. (Top) Layered architecture comprising initial exploration with parallel agents, $K$ recursive enhancement layers propagating reference trajectories as persistent states, and final answer generation. Each layer employs $M_k$ agents operating on structured context $(q,\tau^{(k)},C^{(k)})$ with controlled diversity mechanisms. (Bottom) Refinement operator $R$ workflow: structured trajectory representation, multi-dimensional conflict detection (factual and logical), meta-verification through authority assessment and cross-validation, and segment-level integration of verified improvements. The operator ensures monotonic trajectory improvement via dimensional error minimization while preserving reasoning coherence through hypothesis correction and comparative reflection.
• Figure 2: Evolution of score distributions for generated solutions under RL (left) and MARINE trajectory refinement (right) across iterations. The left panel illustrates how parameter updates in RL shift the entire score distribution to higher values, keeping the probability of exceeding the current pass@1 roughly stable. The right panel shows how conditioning on a fixed, increasingly strong reference trajectory in MARINE biases generation toward a high-score region, thereby enlarging the gap between pass@1 and the reference and reducing the probability that newly sampled trajectories surpass the reference over iterations.
• Figure 3: Performance Comparison. MARINE achieves SOTA with the 685B LLM and matches Kimi-K2 with the 80B LLM.

Theorems & Definitions (5)

• Proposition 4.2: Progressively decreasing probability of successful sampling in MARINE
• Theorem 4.3: Optimal exploring batch size $M_k$ under constrained agent invocation budgets $T$
• proof
• Theorem 4.4: Monotone improvement with growing batch size
• proof