Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction

TL;DR

This work addresses cascading errors in LLM-based multi-agent systems by introducing MASC, a metacognitive framework that performs online, unsupervised, step-level anomaly detection and targeted self-correction. It combines Next-Execution Reconstruction to capture causal dynamics with a Prototype-Guided Enhancement to provide stable references when history is scarce, and it activates a correction agent to revise flagged outputs before propagation. Trained solely on normal trajectories, MASC achieves up to $8.47\%$ relative gains in step-level AUC-ROC on Who&When and delivers consistent end-to-end improvements across six benchmarks and diverse MAS architectures, with an average gain of $1.29\%$ in task accuracy. The approach is plug-and-play, architecture-agnostic, and incurs minimal overhead, offering a practical reliability primitive for scalable, trustworthy multi-agent reasoning with LLMs.

Abstract

Large Language Model based multi-agent systems (MAS) excel at collaborative problem solving but remain brittle to cascading errors: a single faulty step can propagate across agents and disrupt the trajectory. In this paper, we present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction. MASC rethinks detection as history-conditioned anomaly scoring via two complementary designs: (1) Next-Execution Reconstruction, which predicts the embedding of the next step from the query and interaction history to capture causal consistency, and (2) Prototype-Guided Enhancement, which learns a prototype prior over normal-step embeddings and uses it to stabilize reconstruction and anomaly scoring under sparse context (e.g., early steps). When an anomaly step is flagged, MASC triggers a correction agent to revise the acting agent's output before information flows downstream. On the Who&When benchmark, MASC consistently outperforms all baselines, improving step-level error detection by up to 8.47% AUC-ROC ; When plugged into diverse MAS frameworks, it delivers consistent end-to-end gains across architectures, confirming that our metacognitive monitoring and targeted correction can mitigate error propagation with minimal overhead.

Figures (5)

• Figure 1: Comparative analysis of intra-distance, error locations, and metacognitive behavior.
• Figure 2: Overview of MASC. Left: At step $t$, the agent’s output $\mathcal{O}_t$ and history context $\mathcal{H}_{t-1}$ are sent to a real-time detector; if normal, it passes through, otherwise a correction produces $\tilde{\mathcal{O}}_t$, updates $\mathcal{H}_t$, and is used at $t{+}1$. Right: Next-Execution Reconstruction takes projected query $\mathcal{Q}$ and history embeddings, uses a frozen LLM with a learnable head $f_\theta$ to predict $\hat{x}_t$; a prototype $p$ supplies a stability prior, and the anomaly score combines reconstruction error ($d_{L2}$) and prototype misalignment ($d_{cos}$) to trigger self-correction.
• Figure 3: Ablation of reconstruction and prototype modules on Who&When.
• Figure 4: Normal vs. error score distributions on Who&When: MASC (left) vs. BERT (right); MASC shows a cleaner separation.
• Figure 5: Hyperparameter and prototype updating analysis.

Theorems & Definitions (1)

• Definition 1: Error Step Detection in Multi-Agent Systems