Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Experts

TL;DR

The paper addresses the reliability gaps of autonomous web agents operating on live, complex web interfaces, notably grounding accuracy, site-specific procedural knowledge, and long-horizon memory. It introduces Avenir-Web, a modular web agent that combines Mixture of Grounding Experts (MoGE) for robust visual grounding, Experience-Imitation Planning (EIP) for leveraging external procedural knowledge, and a Task-Tracking Checklist with Adaptive Memory to stabilize long-horizon execution. The authors demonstrate open-source state-of-the-art performance on the Online-Mind2Web benchmark, achieving 53.7% Task Success Rate with Gemini 3 Pro and 25.7% with a lightweight 8B model, significantly surpassing open-source baselines and approaching proprietary systems. The work provides a scalable, reproducible framework for reliable web navigation and highlights the value of integrating strategic planning with memory-aware grounding for real-world web interaction.

Abstract

Despite advances in multimodal large language models, autonomous web agents still struggle to reliably execute long-horizon tasks on complex and dynamic web interfaces. Existing agents often suffer from inaccurate element grounding, the absence of site-specific procedural knowledge, and unstable long-term task tracking and memory, particularly when operating over complex Document Object Model structures. To address these limitations, we introduce Avenir-Web, a web agent that achieves a new open-source state of the art on the Online-Mind2Web benchmark in real-world deployment. Avenir-Web leverages a Mixture of Grounding Experts, Experience-Imitation Planning for incorporating procedural priors, and a task-tracking checklist combined with adaptive memory to enable robust and seamless interaction across diverse user interface paradigms. We evaluate Avenir-Web on Online-Mind2Web, a rigorous benchmark of live and user-centered web tasks. Our results demonstrate that Avenir-Web significantly surpasses prior open-source agents and attains performance parity with top-tier proprietary models, thereby establishing a new open-source state of the art for reliable web agents on live websites.

Figures (9)

• Figure 1: Performance of Avenir-Web on the Online-Mind2Webxue2025an benchmark (300 live tasks). The figure compares the success rate of our agent with existing open-source baselines and proprietary models. Avenir-Web achieves a 53.7% success rate, which is shown alongside specialized agents such as ACT-1enhans, Operatoropenai2025introducing_operator, and Navigatornavigator.
• Figure 2: System architecture of Avenir-Web, featuring a decoupled strategic planning and execution framework. The Initialization phase utilizes the Experience-Imitation Planner (EIP) to transform external procedural knowledge into a verifiable Task-Tracking Checklist. During the iterative Execution Loop, the agent maintains strategic consistency through Adaptive Memory while the Mixture of Grounding Experts (MoGE) ensures robust element interaction via hierarchical visual-semantic grounding. A closed-loop feedback mechanism propagates environmental state observations back to the checklist and memory modules to prevent navigational drift in long-horizon tasks.
• Figure 3: Comparison of Experience-Imitation Planning (EIP). Without EIP, the agent executes instructions directly against the live website. With EIP, external how-to knowledge is searched and summarized into a site-specific plan that guides the agent's interaction.
• Figure 4: Avenir-Web execution context for petfinder.com. The figure illustrates the integration of Experience-Imitation Planning (EIP) and Task-Tracking Checklist with status feedback.
• Figure 5: The Task-Tracking Checklist lifecycle. (1) Initialization generates atomic outcome states from the user instruction. (2) During the execution loop, a lightweight model iteratively updates the checklist based on action feedback and environment observations.