SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills

TL;DR

SkillWeaver introduces a three-stage, skill-centric framework for autonomous web agents to self-improve by discovering, practicing, and distilling reusable APIs from website interactions. By converting successful trajectories into Python Playwright APIs and rigorously testing them, the approach yields substantial gains on WebArena and real-world websites, and enables transfer of skills to weaker agents. The results reveal robust improvements, emergent compositional APIs, and meaningful generalization across domains, while also discussing limitations in LLM robustness and safety. Overall, SkillWeaver demonstrates a viable path toward scalable, plug-and-play web automation through self-generated, shareable skill libraries.

Abstract

To survive and thrive in complex environments, humans have evolved sophisticated self-improvement mechanisms through environment exploration, hierarchical abstraction of experiences into reuseable skills, and collaborative construction of an ever-growing skill repertoire. Despite recent advancements, autonomous web agents still lack crucial self-improvement capabilities, struggling with procedural knowledge abstraction, refining skills, and skill composition. In this work, we introduce SkillWeaver, a skill-centric framework enabling agents to self-improve by autonomously synthesizing reusable skills as APIs. Given a new website, the agent autonomously discovers skills, executes them for practice, and distills practice experiences into robust APIs. Iterative exploration continually expands a library of lightweight, plug-and-play APIs, significantly enhancing the agent's capabilities. Experiments on WebArena and real-world websites demonstrate the efficacy of SkillWeaver, achieving relative success rate improvements of 31.8% and 39.8%, respectively. Additionally, APIs synthesized by strong agents substantially enhance weaker agents through transferable skills, yielding improvements of up to 54.3% on WebArena. These results demonstrate the effectiveness of honing diverse website interactions into APIs, which can be seamlessly shared among various web agents.

Figures (10)

• Figure 1: An overview of the SkillWeaver framework. The Skill Proposal module (Stage i@) identifies novel skills to practice based on observations of the environment and available APIs in the skill library. For each proposed skill, the agent executes it to generate trajectories, which are later evaluated by the reward model. If successful, the trajectory is utilized to synthesize an API (Stage ii@). To ensure robustness, the synthesized API undergoes testing with automatically generated test cases and debugging within the Skill Honing module (Stage iii@).
• Figure 2: An example of a synthesized API from https://www.drugs.com/ used to identify pills based on their characteristics.
• Figure 3: Success rate with synthesized vs. human-crafted APIs.
• Figure 4: Express Checkout API execution, before (left) and after (right).
• Figure 5: Apply Discount Code API execution. Before, the agent is on a random page (left), and after, the agent successfully navigates to the shopping cart and enters a discount code, WELCOME10 (right).