Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants

Zuhao Zhang, Chengyue Yu, Yuante Li, Chenyi Zhuang, Linjian Mo, Shuai Li

TL;DR

MiniAppBench is introduced, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation and MiniAppEval, an agentic evaluation framework, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists.

Abstract

With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation. Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists, we propose MiniAppEval, an agentic evaluation framework. Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps, while MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our code is available in github.com/MiniAppBench.

MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants

TL;DR

MiniAppBench is introduced, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation and MiniAppEval, an agentic evaluation framework, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists.

Abstract

With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation. Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists, we propose MiniAppEval, an agentic evaluation framework. Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps, while MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our code is available in github.com/MiniAppBench.
Paper Structure (65 sections, 3 equations, 7 figures, 8 tables, 1 algorithm)

This paper contains 65 sections, 3 equations, 7 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Code Generation and World Reasoning
  4. Web Development
  5. Evaluation Methodologies
  6. MiniAppBench
  7. Overview
  8. Data Representation
  9. Evaluation Dimension
  10. Intention Dimension.
  11. Static Dimension.
  12. Dynamic Dimension.
  13. Dataset Construction Pipeline
  14. ➠ Stage 1: Identifying Principle-Driven Interactive Queries.
  15. ➠ Stage 2: Expanding Coverage While Preserving Core Intent.
  16. ...and 50 more sections

Figures (7)

  • Figure 1: The shift from text to MiniApps. Unlike static text, MiniApps transforms abstract explanations into intuitive visualizations and unlocks actionable tasks (e.g., diet tracking) that were previously impossible.
  • Figure 2: Failure Cases in Principle Adherence.MiniApps require models to capture and instantiate relevant real-world principles, while MiniAppEval proves effective due to its multi-component system design (eval-ref, code, playwright).
  • Figure 3: Overview of the MiniAppBench dataset and construction process. (a)--(d) illustrate the dataset construction pipeline. (e) summarizes the dataset features and distributions (domain and difficulty), with the distribution of subclasses shown in the side bar charts. (f) presents representative MiniApps examples from six domains.
  • Figure 4: MiniAppEval vs. Previous Methods. Unlike brittle scripts or rigid comparisons, MiniAppEval integrates code inspection with dynamic execution. It complements human evaluation by verifying underlying physical principles and automating tedious testing scenarios to ensure robust assessment.
  • Figure 5: Overall model pass rate on MiniAppBench
  • ...and 2 more figures