Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

End-to-End Goal-Driven Web Navigation

Rodrigo Nogueira, Kyunghyun Cho

TL;DR

This work introduces End-to-End Goal-Driven Web Navigation as a large-scale benchmark for evaluating AI agents on natural language understanding and planning within partially observable graph-structured websites. It defines the WebNav framework to convert sites into navigable graphs and WikiNav as a Wikipedia-derived dataset, plus WikiNav-Jeopardy for QA-style queries. The authors propose NeuAgent, a neural navigator with both feedforward and recurrent variants and attention-based query encoding, trained end-to-end via supervised learning and evaluated with beam search; results show strong gains over baselines and clear benefits from history and attention, especially on harder tasks. The study demonstrates the benchmark’s potential for assessing progress in real-world tasks like focused crawling and question answering, and highlights pretrained agents’ advantages for complex navigation.

Abstract

We propose a goal-driven web navigation as a benchmark task for evaluating an agent with abilities to understand natural language and plan on partially observed environments. In this challenging task, an agent navigates through a website, which is represented as a graph consisting of web pages as nodes and hyperlinks as directed edges, to find a web page in which a query appears. The agent is required to have sophisticated high-level reasoning based on natural languages and efficient sequential decision-making capability to succeed. We release a software tool, called WebNav, that automatically transforms a website into this goal-driven web navigation task, and as an example, we make WikiNav, a dataset constructed from the English Wikipedia. We extensively evaluate different variants of neural net based artificial agents on WikiNav and observe that the proposed goal-driven web navigation well reflects the advances in models, making it a suitable benchmark for evaluating future progress. Furthermore, we extend the WikiNav with question-answer pairs from Jeopardy! and test the proposed agent based on recurrent neural networks against strong inverted index based search engines. The artificial agents trained on WikiNav outperforms the engined based approaches, demonstrating the capability of the proposed goal-driven navigation as a good proxy for measuring the progress in real-world tasks such as focused crawling and question-answering.

End-to-End Goal-Driven Web Navigation

TL;DR

This work introduces End-to-End Goal-Driven Web Navigation as a large-scale benchmark for evaluating AI agents on natural language understanding and planning within partially observable graph-structured websites. It defines the WebNav framework to convert sites into navigable graphs and WikiNav as a Wikipedia-derived dataset, plus WikiNav-Jeopardy for QA-style queries. The authors propose NeuAgent, a neural navigator with both feedforward and recurrent variants and attention-based query encoding, trained end-to-end via supervised learning and evaluated with beam search; results show strong gains over baselines and clear benefits from history and attention, especially on harder tasks. The study demonstrates the benchmark’s potential for assessing progress in real-world tasks like focused crawling and question answering, and highlights pretrained agents’ advantages for complex navigation.

Abstract

We propose a goal-driven web navigation as a benchmark task for evaluating an agent with abilities to understand natural language and plan on partially observed environments. In this challenging task, an agent navigates through a website, which is represented as a graph consisting of web pages as nodes and hyperlinks as directed edges, to find a web page in which a query appears. The agent is required to have sophisticated high-level reasoning based on natural languages and efficient sequential decision-making capability to succeed. We release a software tool, called WebNav, that automatically transforms a website into this goal-driven web navigation task, and as an example, we make WikiNav, a dataset constructed from the English Wikipedia. We extensively evaluate different variants of neural net based artificial agents on WikiNav and observe that the proposed goal-driven web navigation well reflects the advances in models, making it a suitable benchmark for evaluating future progress. Furthermore, we extend the WikiNav with question-answer pairs from Jeopardy! and test the proposed agent based on recurrent neural networks against strong inverted index based search engines. The artificial agents trained on WikiNav outperforms the engined based approaches, demonstrating the capability of the proposed goal-driven navigation as a good proxy for measuring the progress in real-world tasks such as focused crawling and question-answering.

Paper Structure

This paper contains 21 sections, 11 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Goal-driven Web Navigation
  3. Constraints
  4. WebNav: Software
  5. WikiNav: A Benchmark Task
  6. Related Work: Wikispeedia
  7. WikiNav-Jeopardy: Jeopardy! on WikiNav
  8. NeuAgent: Neural Network based Agent
  9. Model Description
  10. Core Function
  11. Content Representation
  12. Query Representation
  13. Inference: Beam Search
  14. Training: Supervised Learning
  15. Human Evaluation
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Graphical illustration of a single step performed by the baseline model, NeuAgent.
  • Figure 2: Visualization of the attention weights over a test query. The horizontal axis corresponds to the query words, and the vertical axis to the articles titles visited.