Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Risk-Sensitive Agent Compositions

Guruprerana Shabadi, Rajeev Alur

TL;DR

This work addresses risk-aware composition of agentic workflows by modeling them as agent graphs and optimizing tail risk of safety-, fairness-, and privacy-related losses. It introduces a BucketedVaR dynamic-programming algorithm that allocates a risk budget across edges and uses a union bound to estimate the VaR of a path, with asymptotic near-optimality under edge independence. The method is validated on reinforcement learning benchmarks involving long-horizon tasks like safety navigation and resource use, demonstrating accurate VaR approximation and correct path selection. By focusing on tail risk rather than expected performance, the work advances reliable, risk-aware design of multi-agent systems and points to extensions to CVaR, online bandits, and hardware-accelerated implementations.

Abstract

From software development to robot control, modern agentic systems decompose complex objectives into a sequence of subtasks and choose a set of specialized AI agents to complete them. We formalize agentic workflows as directed acyclic graphs, called agent graphs, where edges represent AI agents and paths correspond to feasible compositions of agents. Real-world deployment requires selecting agent compositions that not only maximize task success but also minimize violations of safety, fairness, and privacy requirements which demands a careful analysis of the low-probability (tail) behaviors of compositions of agents. In this work, we consider risk minimization over the set of feasible agent compositions and seek to minimize the value-at-risk of the loss distribution of the agent composition where the loss quantifies violations of these requirements. We introduce an efficient algorithm which traverses the agent graph and finds a near-optimal composition of agents. It uses a dynamic programming approach to approximate the value-at-risk of agent compositions by exploiting a union bound. Furthermore, we prove that the approximation is near-optimal asymptotically for a broad class of practical loss functions. To evaluate our framework, we consider a suite of video game-like control benchmarks that require composing several agents trained with reinforcement learning and demonstrate our algorithm's effectiveness in approximating the value-at-risk and identifying the optimal agent composition.

Risk-Sensitive Agent Compositions

TL;DR

This work addresses risk-aware composition of agentic workflows by modeling them as agent graphs and optimizing tail risk of safety-, fairness-, and privacy-related losses. It introduces a BucketedVaR dynamic-programming algorithm that allocates a risk budget across edges and uses a union bound to estimate the VaR of a path, with asymptotic near-optimality under edge independence. The method is validated on reinforcement learning benchmarks involving long-horizon tasks like safety navigation and resource use, demonstrating accurate VaR approximation and correct path selection. By focusing on tail risk rather than expected performance, the work advances reliable, risk-aware design of multi-agent systems and points to extensions to CVaR, online bandits, and hardware-accelerated implementations.

Abstract

From software development to robot control, modern agentic systems decompose complex objectives into a sequence of subtasks and choose a set of specialized AI agents to complete them. We formalize agentic workflows as directed acyclic graphs, called agent graphs, where edges represent AI agents and paths correspond to feasible compositions of agents. Real-world deployment requires selecting agent compositions that not only maximize task success but also minimize violations of safety, fairness, and privacy requirements which demands a careful analysis of the low-probability (tail) behaviors of compositions of agents. In this work, we consider risk minimization over the set of feasible agent compositions and seek to minimize the value-at-risk of the loss distribution of the agent composition where the loss quantifies violations of these requirements. We introduce an efficient algorithm which traverses the agent graph and finds a near-optimal composition of agents. It uses a dynamic programming approach to approximate the value-at-risk of agent compositions by exploiting a union bound. Furthermore, we prove that the approximation is near-optimal asymptotically for a broad class of practical loss functions. To evaluate our framework, we consider a suite of video game-like control benchmarks that require composing several agents trained with reinforcement learning and demonstrate our algorithm's effectiveness in approximating the value-at-risk and identifying the optimal agent composition.

Paper Structure

This paper contains 20 sections, 2 theorems, 16 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Risk Minimization Over Agent Graphs
  4. Finding the path minimizes the value-at-risk
  5. Experimental evaluation
  6. Benchmarks
  7. Experimental results
  8. Limitations
  9. Conclusion and future directions
  10. Proofs of correctness, efficiency, and optimality of the BucketedVaR algorithm
  11. Benchmark environments and experimental evaluation setups
  12. 16-Rooms
  13. Environment and agent graph.
  14. Additional experimental observations.
  15. Fetch
  16. ...and 5 more sections

Key Result

Theorem 1

Consider $G = (V, E, X, T, F, L, s, t, \mathcal{D}_s)$ an agent graph, $d \in \mathbb{N}_{>0}$ the number of buckets, $n \in \mathbb{N}_{>0}$ the sample size, and $\alpha \in (0, 1)$ the risk budget. Then, let $q \in \mathbb{R}$ and $p \in V^*$ be the value-at-risk estimate and path returned by $alg Furthermore, the time complexity of $alg:bucketed-var(G, d, n, \alpha)$ is $O(n(d+1)^2\lvert V\rver

Figures (6)

  • Figure 1: An agent graph
  • Figure 2: Top-view map with agent graph of the BoxRelay environment.
  • Figure 3: Absolute quantile difference in percentage between the quantile computed by the \ref{['alg:bucketed-var']} algorithm and the desired quantile. The empirical quantiles are computed on a fresh set of $10^4$ samples.
  • Figure 4: Empirical quantiles of the $\textnormal{VaR}_{0.1}$ estimates computed by \ref{['alg:bucketed-var']} with varying parameters. $95\%$ Clopper-Pearson CIs for the empirical quantiles computed on $10^4$ samples are also plotted.
  • Figure 5: 16-Rooms environment with obstacles to avoid in red. The light blue square in the bottom left corner is the initial room of the point mass. The first subgoal is the dark blue square, followed by green, pink, and brown. Image taken from jothimurugan21:dirl.
  • ...and 1 more figures

Theorems & Definitions (7)

  • Definition : Agent graph
  • Theorem 1
  • proof : Proof sketch.
  • Theorem 2
  • proof : Proof sketch.
  • proof : Proof of \ref{['thm:correctness-bucketed-var']}
  • proof : Proof of \ref{['thm:optimality']}