Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sketch-to-Architecture: Generative AI-aided Architectural Design

Pengzhi Li, Baijuan Li, Zhiheng Li

TL;DR

The paper presents a diffusion-model–driven workflow for early-stage architectural design that converts simple sketches into conceptual floorplans and 3D massing, aided by LoRA fine-tuning and ControlNet for controlled generation. It integrates depth estimation to convert floorplans into 3D geometries via Rhino and Grasshopper, enabling end-to-end renderings guided by core architectural terms and text prompts. A key contribution is the ability to perform local edits with masking, allowing targeted modifications while preserving the overall design, thereby speeding ideation. The approach demonstrates a practical AI-assisted design paradigm that complements traditional workflows and points to future integration of architectural standards and multi-perspective consistency into model training.

Abstract

Recently, the development of large-scale models has paved the way for various interdisciplinary research, including architecture. By using generative AI, we present a novel workflow that utilizes AI models to generate conceptual floorplans and 3D models from simple sketches, enabling rapid ideation and controlled generation of architectural renderings based on textual descriptions. Our work demonstrates the potential of generative AI in the architectural design process, pointing towards a new direction of computer-aided architectural design. Our project website is available at: https://zrealli.github.io/sketch2arc

Sketch-to-Architecture: Generative AI-aided Architectural Design

TL;DR

The paper presents a diffusion-model–driven workflow for early-stage architectural design that converts simple sketches into conceptual floorplans and 3D massing, aided by LoRA fine-tuning and ControlNet for controlled generation. It integrates depth estimation to convert floorplans into 3D geometries via Rhino and Grasshopper, enabling end-to-end renderings guided by core architectural terms and text prompts. A key contribution is the ability to perform local edits with masking, allowing targeted modifications while preserving the overall design, thereby speeding ideation. The approach demonstrates a practical AI-assisted design paradigm that complements traditional workflows and points to future integration of architectural standards and multi-perspective consistency into model training.

Abstract

Recently, the development of large-scale models has paved the way for various interdisciplinary research, including architecture. By using generative AI, we present a novel workflow that utilizes AI models to generate conceptual floorplans and 3D models from simple sketches, enabling rapid ideation and controlled generation of architectural renderings based on textual descriptions. Our work demonstrates the potential of generative AI in the architectural design process, pointing towards a new direction of computer-aided architectural design. Our project website is available at: https://zrealli.github.io/sketch2arc
Paper Structure (14 sections, 6 figures)

This paper contains 14 sections, 6 figures.

Table of Contents

  1. Introduction
  2. Related works
  3. Architecture generation tools
  4. AI in architectural design
  5. Method
  6. Preliminaries
  7. Floorplan and 3D massing model generation
  8. Architectural design
  9. Results
  10. Visualization
  11. Global generation
  12. Local editing
  13. Future work
  14. Conclusion

Figures (6)

  • Figure 1: We explore how generative AI technology can be effectively utilized in the early phases of architectural design.
  • Figure 2: We present the workflow of AI-generated architectural design.
  • Figure 3: We show some of the most critical terms in architectural design, covering various architectural styles, architectural types, architectural materials, etc.
  • Figure 4: We show more architectural designs generated from simple sketches by our method.
  • Figure 5: We show many architectural design results. The results are generated based on different architectural design terms, including architectural style and types. More textual details can be found in Figure 3.
  • ...and 1 more figures