WebGen-V Bench: Structured Representation for Enhancing Visual Design in LLM-based Web Generation and Evaluation

TL;DR

WebGen-V tackles the gap between realistic web design and instruction-to-HTML generation by introducing an unbounded, agentic crawling framework, a structured per-section multimodal data representation, and a section-level evaluation protocol. Its two core modules—Crawling and Evaluation—are powered by a unified Processor that decomposes pages into sections with text, images, and metadata, enabling fine-grained generation and feedback. Empirical results across three state-of-the-art LLM backbones show that section-wise refinement improves layout fidelity, text–image alignment, and readability, and that the structured approach better aligns with human judgment than traditional full-page baselines. The framework also demonstrates how existing benchmarks can be transformed into structured multimodal tasks, offering a scalable path toward realistic, design-consistent web generation and evaluation at scale.

Abstract

Witnessed by the recent advancements on leveraging LLM for coding and multimodal understanding, we present WebGen-V, a new benchmark and framework for instruction-to-HTML generation that enhances both data quality and evaluation granularity. WebGen-V contributes three key innovations: (1) an unbounded and extensible agentic crawling framework that continuously collects real-world webpages and can leveraged to augment existing benchmarks; (2) a structured, section-wise data representation that integrates metadata, localized UI screenshots, and JSON-formatted text and image assets, explicit alignment between content, layout, and visual components for detailed multimodal supervision; and (3) a section-level multimodal evaluation protocol aligning text, layout, and visuals for high-granularity assessment. Experiments with state-of-the-art LLMs and ablation studies validate the effectiveness of our structured data and section-wise evaluation, as well as the contribution of each component. To the best of our knowledge, WebGen-V is the first work to enable high-granularity agentic crawling and evaluation for instruction-to-HTML generation, providing a unified pipeline from real-world data acquisition and webpage generation to structured multimodal assessment.

Figures (15)

• Figure 1: Overview of our framework consisting of two core modules, Crawling and Evaluation, which share a generic and scalable Processor for producing and analyzing structured webpage representations. The same Processor can also parse and transform existing benchmarks (e.g., WebMMU), as illustrated by the dashed connections. Evaluation and feedback outputs can be seamlessly integrated with generation models (Gen) to enable iterative refinement and continuous improvement.
• Figure 2: Relative improvements over zero-shot generation across nine metrics for three backbone models: (a) Claude-Opus-4.1, (b) GPT-5, and (c) Gemini-2.5-Pro. Each radar plot shows the normalized gain achieved by one-turn refinement using either non-structured full-page input (blue) or our structured representation (red). Larger areas indicate stronger improvements.
• Figure 3: Head-to-head win-rate comparison averaged across all backbone models (Claude-Opus-4.1, GPT-5, and Gemini-2.5-Pro). Each bar represents the mean proportion of wins, ties, and losses under pairwise evaluation across all model backbones.
• Figure 4: Ablation study on different input modalities. Structured text and section-wise screenshots yield the strongest improvements over the Gemini-2.5 zero-shot generation, particularly for readability, spacing, and media placement. Full-page inputs blur fine-grained errors and often regress.
• Figure 5: Comparison between the WebMMU HTML rendering and our regenerated version. Our processor transforms raw HTML into structured representations, enabling reuse of existing benchmarks in instruction-to-HTML settings.