PlantoGraphy: Incorporating Iterative Design Process into Generative Artificial Intelligence for Landscape Rendering

TL;DR

PlantoGraphy addresses the mismatch between end-to-end GAI landscape rendering and conventional iterative design. It introduces a two-stage pipeline—idea concretization via scene graphs using domain-informed LLM reasoning, followed by layout-guided illustration through a LoRA-finetuned diffusion model with instance-based latent composition. Through modular experiments and a within-subjects user study with expert designers, the approach demonstrates improved graphical coherence, better alignment with design intents, and meaningful time savings, while revealing areas for improvement in plant diversity, bias, and interaction modalities. The work advances human-centered AI design by embedding domain knowledge, enabling interactive graph-based editing, and showing promise for broader applicability across design domains. It offers practical impact for designers seeking iterative AI-assisted landscape renderings that maintain control and communicability with stakeholders.

Abstract

Landscape renderings are realistic images of landscape sites, allowing stakeholders to perceive better and evaluate design ideas. While recent advances in Generative Artificial Intelligence (GAI) enable automated generation of landscape renderings, the end-to-end methods are not compatible with common design processes, leading to insufficient alignment with design idealizations and limited cohesion of iterative landscape design. Informed by a formative study for comprehending design requirements, we present PlantoGraphy, an iterative design system that allows for interactive configuration of GAI models to accommodate human-centered design practice. A two-stage pipeline is incorporated: first, concretization module transforms conceptual ideas into concrete scene layouts with a domain-oriented large language model; and second, illustration module converts scene layouts into realistic landscape renderings using a fine-tuned low-rank adaptation diffusion model. PlantoGraphy has undergone a series of performance evaluations and user studies, demonstrating its effectiveness in landscape rendering generation and the high recognition of its interactive functionality.

Figures (14)

• Figure 1: Comparison of traditional and AI-aided design process for landscape rendering. Various AIGC models can be applied to different stages, yet these methods operate in an end-to-end manner without integrating iterative design.
• Figure 2: Overview of the system workflow. PlantoGraphy incorporates a two-stage framework to transform design concepts described in textual content into realistic images of landscape rendering. First, the concentization module leverages a graph-enhanced LLM to transform design descriptions into layouts, using scene graphs to improve the comprehension of user input. Next, the illustration module employs a fine-tuned LoRA model to generate realistic landscape renderings based on the layout.
• Figure 3: Interactive visual interface for PlantoGraphy. Users can input textual descriptions of the scene in the Text Panel and customize the design by manipulating the scene graph in the Graph Panel and updating the layout in the Layout Panel. The rendering results are presented in the Rendering Panel.
• Figure 4: Framework of idea concretization via expert-engaged interaction with LLMs for scene graph generator and layout generator.
• Figure 5: Prompt template for landscape scene layout generation. The template consists of four main components: task description, constraints, contextual information, and demonstrations.