Textual Equilibrium Propagation for Deep Compound AI Systems

TL;DR

The paper addresses the problem that global textual backpropagation in deep compound AI pipelines suffers from depth-dependent gradient pathologies, namely exploding and vanishing textual gradients. It introduces Textual Equilibrium Propagation (TEP), a local, two-phase learning framework with free-phase equilibrium and nudged-phase bounded prompt edits, achieving depth-robust updates without long backprop chains. The method is formalized via stochastic computation graphs and analyzed for convergence, while extensive experiments on long-horizon QA and multi-agent tool-use benchmarks show TEP consistently outperforms TextGrad and related baselines, with gains growing as depth increases. The results demonstrate that local equilibrium-based, bounded updates can effectively coordinate complex AI systems in a scalable and practical manner, preserving the practicality of black-box LLM components while improving accuracy and efficiency.

Abstract

Large language models (LLMs) are increasingly deployed as part of compound AI systems that coordinate multiple modules (e.g., retrievers, tools, verifiers) over long-horizon workflows. Recent approaches that propagate textual feedback globally (e.g., TextGrad) make it feasible to optimize such pipelines, but we find that performance degrades as system depth grows. In particular, long-horizon agentic workflows exhibit two depth-scaling failure modes: 1) exploding textual gradient, where textual feedback grows exponentially with depth, leading to prohibitively long message and amplifies evaluation biases; and 2) vanishing textual gradient, where limited long-context ability causes models overemphasize partial feedback and compression of lengthy feedback causes downstream messages to lose specificity gradually as they propagate many hops upstream. To mitigate these issues, we introduce Textual Equilibrium Propagation (TEP), a local learning principle inspired by Equilibrium Propagation in energy-based models. TEP includes two phases: 1) a free phase where a local LLM critics iteratively refine prompts until reaching equilibrium (no further improvements are suggested); and 2) a nudged phase which applies proximal prompt edits with bounded modification intensity, using task-level objectives that propagate via forward signaling rather than backward feedback chains. This design supports local prompt optimization followed by controlled adaptation toward global goals without the computational burden and signal degradation of global textual backpropagation. Across long-horizon QA benchmarks and multi-agent tool-use dataset, TEP consistently improves accuracy and efficiency over global propagation methods such as TextGrad. The gains grows with depth, while preserving the practicality of black-box LLM components in deep compound AI system.

Figures (6)

• Figure 1: Textual gradient failure modes in deep compound AI systems, an example bug report propagates through a multi-agent code generation pipeline (Problem Analyzer → Code Generator → Test Generator → Final Validator). Top--Exploding gradients: Each agent adds context to preserve information, causing exponential token growth that buries the actionable one-line fix beneath layers of accumulated texts. Bottom--Vanishing gradients: Compression to manage gradient explosion strips away critical specifics (file paths, line numbers, test cases), leaving only generic, non-actionable advice.
• Figure 2: Overview of Textual Equilibrium Propagation (TEP). (a) TextGrad: Multi-step workflows as Stochastic Computation Graphs (SCGs) where nodes are LLM agents with configurable prompts and edges represent data flow. Global textual backpropagation suffers exploding gradients (exponentially growing feedback) and vanishing gradients (decaying specificity) at depth. (b) Textual Equilibrium Propagation: Free phase optimizes each node locally until no further improvements are suggested (equilibrium), then nudged phase applies bounded prompt modifications guided by task objectives. This local approach avoids global feedback chains.
• Figure 3: Textual gradient failure modes with increasing workflow depth on BigCodeBench. (a) Exploding gradients: TextGrad feedback messages grow exponentially, reaching context overflow at scale factor $5$. (b) Vanishing gradients: Compressed feedback (TextGrad w/ Sum) loses specificity, leading to sharp drop in effective update rates. TEP maintains stable performance across all scales through local optimization.
• Figure 4: Component ablation analysis demonstrating the necessity of both TEP phases across task types.
• Figure 5: TEP convergence analysis on PubMedQA. (a) Free phase local critic scoring over 20 iterations on 0-10 scale, showing characteristic fluctuations during equilibrium-seeking toward threshold of 8.0. (b) Global convergence across 40 TEP iterations with steady fluctuations from iteration 25 onwards, demonstrating cumulative performance improvements through repeated optimization cycles.