From PowerPoint UI Sketches to Web-Based Applications: Pattern-Driven Code Generation for GIS Dashboard Development Using Knowledge-Augmented LLMs, Context-Aware Visual Prompting, and the React Framework

TL;DR

This work tackles the heavy manual labor involved in building CyberGIS dashboards by integrating a knowledge-augmented generation (KAG) framework with context-aware visual prompting and Retrieval-Augmented Generation (RAG) to translate non-code wireframes into scalable front-end GIS apps. It combines a Python-based Visual Contextual Prompting pipeline, a knowledge-graph-backed knowledge base, and iterative code generation to produce React-based MVVM-structured front-ends that comply with industry patterns. Case studies on meteorological data and wind-energy visualization demonstrate automatic generation of modular dashboards with time-series, shapefiles, and rasters, while maintaining data integrity through embedded quality checks. The approach reduces development time and increases maintainability and scalability for smart city GIS tools, providing a practical route for domain scientists to create production-grade dashboards with limited software engineering training. It advances AI-assisted software development by merging CoT reasoning, RAG, and domain knowledge into geospatial visualization front-ends.

Abstract

Developing web-based GIS applications, commonly known as CyberGIS dashboards, for querying and visualizing GIS data in environmental research often demands repetitive and resource-intensive efforts. While Generative AI offers automation potential for code generation, it struggles with complex scientific applications due to challenges in integrating domain knowledge, software engineering principles, and UI design best practices. This paper introduces a knowledge-augmented code generation framework that retrieves software engineering best practices, domain expertise, and advanced technology stacks from a specialized knowledge base to enhance Generative Pre-trained Transformers (GPT) for front-end development. The framework automates the creation of GIS-based web applications (e.g., dashboards, interfaces) from user-defined UI wireframes sketched in tools like PowerPoint or Adobe Illustrator. A novel Context-Aware Visual Prompting method, implemented in Python, extracts layouts and interface features from these wireframes to guide code generation. Our approach leverages Large Language Models (LLMs) to generate front-end code by integrating structured reasoning, software engineering principles, and domain knowledge, drawing inspiration from Chain-of-Thought (CoT) prompting and Retrieval-Augmented Generation (RAG). A case study demonstrates the framework's capability to generate a modular, maintainable web platform hosting multiple dashboards for visualizing environmental and energy data (e.g., time-series, shapefiles, rasters) from user-sketched wireframes. By employing a knowledge-driven approach, the framework produces scalable, industry-standard front-end code using design patterns such as Model-View-ViewModel (MVVM) and frameworks like React. This significantly reduces manual effort in design and coding, pioneering an automated and efficient method for developing smart city software.

Figures (8)

• Figure 1: From annotated wireframe to code generation, a knowledge driven framework for automated software development of cyberGIS platform for visualizing time-series and spatial data.
• Figure 2: Overall framework architecture for LLM-driven code generation using knowledge-augmented prompting and structured domain knowledge.
• Figure 3: Structured knowledge representations for converting plain-language annotations from UI wireframes into structured prompts with technical terminology.
• Figure 4: A procedure-based, knowledge-augmented code generation process that aligns with the conventions and best practices of the React framework.
• Figure 5: The UI wireframe for the homepage of the web-based application includes thumbnails that serve as navigation links, directing users to the dashboard generated for Use Case I.