FlowAgent: Achieving Compliance and Flexibility for Workflow Agents
Yuchen Shi, Siqi Cai, Zihan Xu, Yuei Qin, Gang Li, Hang Shao, Jiawei Chen, Deqing Yang, Ke Li, Xing Sun
TL;DR
FlowAgent addresses the challenge of enforcing procedural workflows while preserving the flexibility of LLM-driven dialogue. It introduces a Procedure Description Language (PDL) to express hybrid natural language and code workflows and a controller-based FlowAgent to supervise out-of-workflow handling within a DAG-based workflow model $\mathcal{G}(\mathcal{V},\mathcal{E})$ and an MDPlike decision process. The work contributes (i) PDL syntax and semantics, (ii) a FlowAgent execution framework with pre- and post-decision controllers, and (iii) an evaluation benchmark across SGD, STAR, and In-house datasets demonstrating strong compliance and robust flexibility, with ablations confirming controller necessity. These results suggest FlowAgent provides a practical framework for reliable, adaptable LLM-based workflow automation in real-world settings, balancing governance with user-centric interaction.
Abstract
The integration of workflows with large language models (LLMs) enables LLM-based agents to execute predefined procedures, enhancing automation in real-world applications. Traditional rule-based methods tend to limit the inherent flexibility of LLMs, as their predefined execution paths restrict the models' action space, particularly when the unexpected, out-of-workflow (OOW) queries are encountered. Conversely, prompt-based methods allow LLMs to fully control the flow, which can lead to diminished enforcement of procedural compliance. To address these challenges, we introduce FlowAgent, a novel agent framework designed to maintain both compliance and flexibility. We propose the Procedure Description Language (PDL), which combines the adaptability of natural language with the precision of code to formulate workflows. Building on PDL, we develop a comprehensive framework that empowers LLMs to manage OOW queries effectively, while keeping the execution path under the supervision of a set of controllers. Additionally, we present a new evaluation methodology to rigorously assess an LLM agent's ability to handle OOW scenarios, going beyond routine flow compliance tested in existing benchmarks. Experiments on three datasets demonstrate that FlowAgent not only adheres to workflows but also effectively manages OOW queries, highlighting its dual strengths in compliance and flexibility. The code is available at https://github.com/Lightblues/FlowAgent.