DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models

TL;DR

This paper tackles the problem of design-quality gaps in frontend UIs produced by large language models. It introduces DesignRepair, a dual-stream system that leverages Material Design 3 as a knowledge base, constructing a Component KB (KB-Comp) and a System Design KB (KB-System) to guide automated detection and repair of UI design issues in both code and rendered output. The approach combines offline knowledge-base construction, online page extraction, and knowledge-driven repair via retrieval-augmented generation, validated on AI-generated V0 projects and GitHub frontends, with a user study confirming improved perceived quality. Results show high recall and precision across detection and repair stages, and statistically significant improvements in user satisfaction, underscoring the method’s potential to elevate UI quality in LLM-assisted frontend development.

Abstract

The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's V0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics.

Figures (8)

• Figure 1: "URL Shortener" by Vercel's V0: Initial Design vs. Improved Design.
• Figure 2: Examples of recommended and don't design guidelines in Material Design 3. Both guidelines are related to Button component in the Components section, and regard to the Usage aspect. We categorised the recommended guideline as a soft constraint, and do/don't guidelines as hard constraints.
• Figure 3: Framework Overview of Our Approach
• Figure 4: Distribution of Component Guidelines
• Figure 5: Examples of Component and System Design Knowledge entities