CTHA: Constrained Temporal Hierarchical Architecture for Stable Multi-Agent LLM Systems

TL;DR

CTHA addresses instability in multi-time-scale LLM agent systems by constraining inter-layer communication to structured manifolds and introducing a learned Arbiter to guarantee conflict-free composition. It formalizes three mechanisms—Message Contracts, Authority Manifolds, and Arbiter Resolution—creating non-expansive, closed, and deterministic interactions across temporal layers. Empirically, CTHA yields strong gains across tool use, web navigation, SWE, and long-horizon planning benchmarks, including a 47% reduction in coordination failures and 2.3× sample efficiency, with modest latency overhead thanks to selective activation and parallel execution. The approach emphasizes reproducibility through open-source models and demonstrates robust generalization across model families and task horizons. Overall, CTHA offers a principled path to scalable, stable autonomous LLM agents that leverage temporal abstraction without sacrificing reliability or safety.

Abstract

Recently, multi-time-scale agent architectures have extended the ubiquitous single-loop paradigm by introducing temporal hierarchies with distinct cognitive layers. While yielding substantial performance gains, this diversification fundamentally compromises the coordination stability intrinsic to unified agent systems, which causes severe inter-layer conflicts, unbounded error propagation, and restricted scalability. To address these challenges, we propose Constrained Temporal Hierarchical Architecture (CTHA), a general framework that projects the inter-layer communication space onto structured manifolds to restore coordination stability, while incorporating principled arbitration mechanisms to ensure coherent decision-making. Specifically, CTHA enforces three key constraints: (1) Message Contract Constraints that formalize information flow between layers via typed summary, plan, and policy packets; (2) Authority Manifold Constraints that bound each layer's decision space according to its temporal scope; and (3) Arbiter Resolution Constraints that guarantee conflict-free composition of multi-layer decisions. Empirical experiments demonstrate that CTHA is effective for complex task execution at scale, offering 47% reduction in failure cascades, 2.3x improvement in sample efficiency, and superior scalability compared to unconstrained hierarchical baselines. We anticipate that CTHA, as a principled extension of temporal hierarchies, will contribute to a deeper understanding of multi-agent coordination and suggest promising directions for the evolution of robust autonomous systems.

Figures (8)

• Figure 1: Illustrations of Agent Architecture Paradigms. This figure compares the structural design of (a) standard Single-Scale Agent, (b) unconstrained Temporal Hierarchy (TH), and (c) our proposed Constrained Temporal Hierarchical Architecture (CTHA). Unlike the unconstrained TH, CTHA focuses on optimizing the inter-layer communication space by projecting messages onto constrained manifolds ($\mathcal{P}_{\mathcal{M}^{sum}}$, $\mathcal{P}_{\mathcal{M}^{plan}}$, $\mathcal{P}_{\mathcal{M}^{pol}}$) and routing decisions through a principled Arbiter to ensure stability.
• Figure 2: Coordination Instability of Unconstrained Temporal Hierarchy. (a) Task success rate over extended execution. Unconstrained TH exhibits performance collapse around step 150, while constrained variants maintain stable improvement. (b) Amax Gain Magnitude of the composite mapping $\prod_i \mathcal{H}^{\text{res}}_i$ across layers. Unconstrained mappings show exponential growth exceeding $10^3$, while constrained mappings remain bounded near the ideal value of 1. Results averaged over 100 task episodes.
• Figure 3: Message Contract Schemas. CTHA defines three message types with strict field specifications. Summary messages flow upward (fast to slow layers), Plan messages flow downward (slow to fast layers), and Policy messages broadcast from the Institutional layer to all others. Each field has explicit type constraints and cardinality limits.
• Figure 4: Selective Layer Activation Pattern. The Reflex layer activates every step, Tactical every $k_1=3$ steps, Strategic on goal completion events, and Institutional only at session boundaries. This reduces average active layers from 4.0 to 1.8 per step.
• Figure 5: Complete CTHA Architecture. The four temporal layers (center, orange) are instantiated with heterogeneous open-source LLMs selected based on layer requirements. DeepSeek-V3.2-Speciale handles complex reasoning at the Institutional layer, Kimi-K2 manages strategic planning, Qwen3-32B executes tactical decisions, and GLM-4.6-9B provides fast reflexive responses. Message Contracts (purple) constrain inter-layer communication through typed schemas. The Arbiter (red) resolves conflicts and produces the final action sent to the environment. Authority Manifolds (dashed boundary) constrain each layer's decision scope.