Web-CogReasoner: Towards Knowledge-Induced Cognitive Reasoning for Web Agents

TL;DR

This work addresses the challenge of enabling web agents to perform cognitively robust reasoning by grounding knowledge acquisition in a Bloom-inspired two-phase framework (Factual/Conceptual first, Procedural second). It introduces Web-CogKnowledge and a three-tier knowledge taxonomy, builds the large Web-CogDataset and Web-CogBench, and trains Web-CogReasoner with a Knowledge-driven Chain-of-Thought that maps prompts to plans and actions. Empirical results across Web-CogBench, VisualWebBench, and online tasks show superior performance and generalization, with ablations confirming the necessity of each knowledge layer and the KCoT mechanism. The authors release code and data publicly to foster further research in knowledge-grounded, multimodal web agents.

Abstract

Multimodal large-scale models have significantly advanced the development of web agents, enabling perception and interaction with digital environments akin to human cognition. In this paper, we argue that web agents must first acquire sufficient knowledge to effectively engage in cognitive reasoning. Therefore, we decompose a web agent's capabilities into two essential stages: knowledge content learning and cognitive processes. To formalize this, we propose Web-CogKnowledge Framework, categorizing knowledge as Factual, Conceptual, and Procedural. In this framework, knowledge content learning corresponds to the agent's processes of Memorizing and Understanding, which rely on the first two knowledge types, representing the "what" of learning. Conversely, cognitive processes correspond to Exploring, grounded in Procedural knowledge, defining the "how" of reasoning and action. To facilitate knowledge acquisition, we construct the Web-CogDataset, a structured resource curated from 14 real-world websites, designed to systematically instill core knowledge necessary for web agent. This dataset serves as the agent's conceptual grounding-the "nouns" upon which comprehension is built-as well as the basis for learning how to reason and act. Building on this foundation, we operationalize these processes through a novel knowledge-driven Chain-of-Thought (CoT) reasoning framework, developing and training our proposed agent, the Web-CogReasoner. Extensive experimentation reveals its significant superiority over existing models, especially in generalizing to unseen tasks where structured knowledge is decisive. To enable rigorous evaluation, we introduce the Web-CogBench, a comprehensive evaluation suite designed to assess and compare agent performance across the delineated knowledge domains and cognitive capabilities. Our code and data is open sourced at https://github.com/Gnonymous/Web-CogReasoner

Figures (7)

• Figure 1: Visualization of the Web-CogKnowledge Framework along with the experimental results.
• Figure 2: Illustration of Web-CogDataset.
• Figure 3: Illustrating the Knowledge-driven Chain of Thought (CoT) process.
• Figure 4: How people handle unfamiliar web pages. People learn factual, conceptual, and procedural knowledge to memorize, understand, and explore the web, ultimately completing specific tasks.
• Figure 5: Statistics of selected websites by category.