ReUseIt: Synthesizing Reusable AI Agent Workflows for Web Automation

TL;DR

ReUseIt tackles unreliable AI-powered web agents performing repetitive tasks by automatically synthesizing reusable workflows from a history of successful and failed executions. The synthesized workflows include execution guards with pre-/post-condition checks and fallback actions to guide agent behavior, improve self-recovery, and surface failures for user inspection. Benchmark results across 15 task families show the average success rate rising from 24.2% (baseline) to 70.1% with ReUseIt, while a user study demonstrates easier failure understanding and higher trust. The work also investigates user interaction patterns, showing that execution guards reduce guidance burden and enhance interpretability, with potential for scaling via a RAG-based workflow-unit library. Overall, ReUseIt demonstrates a practical path to reliable, reusable web automation through automated guard synthesis and human-in-the-loop feedback.

Abstract

AI-powered web agents have the potential to automate repetitive tasks, such as form filling, information retrieval, and scheduling, but they struggle to reliably execute these tasks without human intervention, requiring users to provide detailed guidance during every run. We address this limitation by automatically synthesizing reusable workflows from an agent's successful and failed attempts. These workflows incorporate execution guards that help agents detect and fix errors while keeping users informed of progress and issues. Our approach enables agents to successfully complete repetitive tasks of the same type with minimal intervention, increasing the success rates from 24.2% to 70.1% across fifteen tasks. To evaluate this approach, we invited nine users and found that our agent helped them complete web tasks with a higher success rate and less guidance compared to two baseline methods, as well as allowed users to easily monitor agent behavior and understand failures.

Figures (9)

• Figure 1: Given a user instruction, the web agent can make low-level but critical mistakes that prevent it from completing tasks. For example, the agents mistakenly used the '+' next to children to add adults on Booking.com (left) and selected 'round-trip' from the drop-down menu despite users asking for 'one-way' flights on Google Flights (right).
• Figure 2: Examples of Success-Traces (A), Magentic-UI Plan (B), and Magentic-UI Plan User (C; user edits are shown in blue text).
• Figure 3: Evaluation of web agents on six web automation tasks and their variations (avg $\pm$ std success rate).
• Figure 4: Automatic workflow synthesis pipeline. From multiple task executions of tasks with variations, ReUseIt extracts agent challenges and successful experiences from failed and successful executions, respectively. Agent challenges are turned into condition checks at failed steps, while successful experiences contribute to fallback actions for agent self-recovery. Condition checks and fallback actions are concatenated with the corresponding step to form a unit in the synthesized workflow. Note that specific values are turned into generic framing in the workflow to support task generalizability.
• Figure 5: Agent execution guided by the synthesized workflow. At actions where pre-/post-condition checks exist in the workflow, ReUseIt captures the webpage screenshot and does a Q&A to determine if the condition check is met before/after this action. If the answer is 'Yes', the agent proceeds to the next action, but if not, it retries executing this action according to the fallback actions. The agent moves on the the next action if the condition check is met after retries. However, if the retries fail, ReUseIt notifies the user by providing a challenge analysis to help them understand the challenge and provide guidance. User guidance is used to iteratively update the condition checks and fallback actions in the workflow.