Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Frontend Diffusion: Exploring Intent-Based User Interfaces through Abstract-to-Detailed Task Transitions

Qinshi Zhang, Latisha Besariani Hendra, Mohan Chi, Zijian Ding

TL;DR

The paper addresses the challenge of bridging abstract user intent and concrete frontend implementations within command-heavy interfaces. It proposes Frontend Diffusion, an end-to-end, LLM-powered pipeline that maps sketches to high-quality websites via a three-stage abstract-to-detailed task transition: sketching, writing, and coding. Key contributions include a model-agnostic design, a PRD generation component, an SVG-to-JPG preprocessing step to optimize LLM input, and an iterative refinement loop that produces multiple ready-to-use code versions. The approach demonstrates potential to reduce human intervention and communication costs in frontend development and suggests extensions to other interdependent workflows, such as video production, thereby advancing intent-based UI paradigms with practical implications for scalable, tool-supported design pipelines.

Abstract

The emergence of Generative AI is catalyzing a paradigm shift in user interfaces from command-based to intent-based outcome specification. In this paper, we explore abstract-to-detailed task transitions in the context of frontend code generation as a step towards intent-based user interfaces, aiming to bridge the gap between abstract user intentions and concrete implementations. We introduce Frontend Diffusion, an end-to-end LLM-powered tool that generates high-quality websites from user sketches. The system employs a three-stage task transition process: sketching, writing, and coding. We demonstrate the potential of task transitions to reduce human intervention and communication costs in complex tasks. Our work also opens avenues for exploring similar approaches in other domains, potentially extending to more complex, interdependent tasks such as video production.

Frontend Diffusion: Exploring Intent-Based User Interfaces through Abstract-to-Detailed Task Transitions

TL;DR

The paper addresses the challenge of bridging abstract user intent and concrete frontend implementations within command-heavy interfaces. It proposes Frontend Diffusion, an end-to-end, LLM-powered pipeline that maps sketches to high-quality websites via a three-stage abstract-to-detailed task transition: sketching, writing, and coding. Key contributions include a model-agnostic design, a PRD generation component, an SVG-to-JPG preprocessing step to optimize LLM input, and an iterative refinement loop that produces multiple ready-to-use code versions. The approach demonstrates potential to reduce human intervention and communication costs in frontend development and suggests extensions to other interdependent workflows, such as video production, thereby advancing intent-based UI paradigms with practical implications for scalable, tool-supported design pipelines.

Abstract

The emergence of Generative AI is catalyzing a paradigm shift in user interfaces from command-based to intent-based outcome specification. In this paper, we explore abstract-to-detailed task transitions in the context of frontend code generation as a step towards intent-based user interfaces, aiming to bridge the gap between abstract user intentions and concrete implementations. We introduce Frontend Diffusion, an end-to-end LLM-powered tool that generates high-quality websites from user sketches. The system employs a three-stage task transition process: sketching, writing, and coding. We demonstrate the potential of task transitions to reduce human intervention and communication costs in complex tasks. Our work also opens avenues for exploring similar approaches in other domains, potentially extending to more complex, interdependent tasks such as video production.
Paper Structure (6 sections, 2 figures)

This paper contains 6 sections, 2 figures.

Table of Contents

  1. Introduction
  2. System Design
  3. Sketching: Visual Layout Design and Theme Input
  4. Writing: Product Requirements Document Generation
  5. Coding: Website Generation and Iterative Refinement
  6. Results and Discussion

Figures (2)

  • Figure 1: Task transition paradigm to bridge more abstract, intent-based task expression to more concrete, command-based implementations. The task transition process flows from sketching (sketch and theme inputs) to writing (product requirement document generation) and coding (frontend code generation and iteration).
  • Figure 2: Screenshots of generated websites spanning commercial and academic domains. The theme prompt as user intent to generate each website is shown beneath the corresponding website.