Shaping Realities: Enhancing 3D Generative AI with Fabrication Constraints

TL;DR

Generative AI for 3D design currently yields aesthetically pleasing but potentially non-functional geometry when intended for fabrication. The paper argues for augmentations that enforce functional constraints, proposing two focus areas: preserving functionality during style edits and encoding functionality into designs from scratch. It discusses challenges in identifying functional regions, ensuring material-specific constraints, and integrating simulations and feedback loops within the generative process. The goal is to bridge digital creativity and real-world applicability, enabling fabrication-ready 3D models generated by AI. As a workshop position piece, it outlines concrete directions and potential implications for maker communities and product design.

Abstract

Generative AI tools are becoming more prevalent in 3D modeling, enabling users to manipulate or create new models with text or images as inputs. This makes it easier for users to rapidly customize and iterate on their 3D designs and explore new creative ideas. These methods focus on the aesthetic quality of the 3D models, refining them to look similar to the prompts provided by the user. However, when creating 3D models intended for fabrication, designers need to trade-off the aesthetic qualities of a 3D model with their intended physical properties. To be functional post-fabrication, 3D models have to satisfy structural constraints informed by physical principles. Currently, such requirements are not enforced by generative AI tools. This leads to the development of aesthetically appealing, but potentially non-functional 3D geometry, that would be hard to fabricate and use in the real world. This workshop paper highlights the limitations of generative AI tools in translating digital creations into the physical world and proposes new augmentations to generative AI tools for creating physically viable 3D models. We advocate for the development of tools that manipulate or generate 3D models by considering not only the aesthetic appearance but also using physical properties as constraints. This exploration seeks to bridge the gap between digital creativity and real-world applicability, extending the creative potential of generative AI into the tangible domain.