How to Correctly do Semantic Backpropagation on Language-based Agentic Systems

TL;DR

The paper addresses optimizing language-based agentic systems modeled as graphs (GASO) by introducing semantic gradients and semantic backpropagation to solve credit-assignment challenges. It unifies reverse-mode differentiation with semantically meaningful gradients, using neighborhood-conditioned backward mappings and an LLM-driven update procedure (semantic gradient descent) with an update gate for stability. Empirical results on BBH, GSM8K, and LIAR demonstrate improved performance and robustness compared to TextGrad, OptoPrime, and COPRO, while highlighting the importance of neighborhood information and gating. Overall, the approach reduces manual optimization and enhances scalability for complex GASO deployments, with an open-source implementation available.

Abstract

Language-based agentic systems have shown great promise in recent years, transitioning from solving small-scale research problems to being deployed in challenging real-world tasks. However, optimizing these systems often requires substantial manual labor. Recent studies have demonstrated that these systems can be represented as computational graphs, enabling automatic optimization. Despite these advancements, most current efforts in Graph-based Agentic System Optimization (GASO) fail to properly assign feedback to the system's components given feedback on the system's output. To address this challenge, we formalize the concept of semantic backpropagation with semantic gradients -- a generalization that aligns several key optimization techniques, including reverse-mode automatic differentiation and the more recent TextGrad by exploiting the relationship among nodes with a common successor. This serves as a method for computing directional information about how changes to each component of an agentic system might improve the system's output. To use these gradients, we propose a method called semantic gradient descent which enables us to solve GASO effectively. Our results on both BIG-Bench Hard and GSM8K show that our approach outperforms existing state-of-the-art methods for solving GASO problems. A detailed ablation study on the LIAR dataset demonstrates the parsimonious nature of our method. A full copy of our implementation is publicly available at https://github.com/HishamAlyahya/semantic_backprop

Figures (19)

• Figure 1: The entire process of our proposed LLM-based solution to GASO. Given a sample query to optimize over, (1) the forward pass of each node can be executed by joining an instruction alongside other inputs to process. Then, (2,3) the semantic gradients are generated through semantic backpropagation that crucially takes into account the neighboring nodes. And finally, (4), the semantic gradients accumulated are joined with the optimizable parameter (e.g., the instruction) and an optimization meta-prompt to retrieve update in the direction given by the semantic gradients.
• Figure 2: Initial graphs for general question answering on BBH and GSM8K (a) and LIAR (b). The variables in green (the $\theta$s) are optimizable.
• Figure 3: Prompt template of forward function for general question answering
• Figure 4: Prompt template of backward function for general question answering
• Figure 5: Prompt template of forward function for LIAR. Function template_context is for the first five intermediate variables, while function template_final is for the answer variable.