ParSEL: Parameterized Shape Editing with Language

TL;DR

Analytical Edit Propagation (AEP) is introduced, an algorithm which extends a seed edit with additional operations until a complete editing program has been formed and is demonstrated to be effective in enabling controllable editing of 3D objects through natural language requests over alternative system designs.

Abstract

The ability to edit 3D assets from natural language presents a compelling paradigm to aid in the democratization of 3D content creation. However, while natural language is often effective at communicating general intent, it is poorly suited for specifying precise manipulation. To address this gap, we introduce ParSEL, a system that enables controllable editing of high-quality 3D assets from natural language. Given a segmented 3D mesh and an editing request, ParSEL produces a parameterized editing program. Adjusting the program parameters allows users to explore shape variations with a precise control over the magnitudes of edits. To infer editing programs which align with an input edit request, we leverage the abilities of large-language models (LLMs). However, while we find that LLMs excel at identifying initial edit operations, they often fail to infer complete editing programs, and produce outputs that violate shape semantics. To overcome this issue, we introduce Analytical Edit Propagation (AEP), an algorithm which extends a seed edit with additional operations until a complete editing program has been formed. Unlike prior methods, AEP searches for analytical editing operations compatible with a range of possible user edits through the integration of computer algebra systems for geometric analysis. Experimentally we demonstrate ParSEL's effectiveness in enabling controllable editing of 3D objects through natural language requests over alternative system designs.

Figures (7)

• Figure 1: Overview: Given a segmented 3D chair and an edit request to "widen the chair," we first convert the shape into a structured representation: hexahedrons and inter-part relations. This abstraction is illustrated on the left, with symmetry relations annotated on the left and attachment relations depicted with red points on the shape. Our neuro-symbolic approach (center) uses an LLM, to interpret the natural language input, and Analytical Edit Propagation, to perform geometric reasoning, in order to infer a parameterized editing program (right) that aligns with the edit request.
• Figure 2: Edit Propagation: Starting with a seed-edit, scaling the seat, new edits are incrementally introduced to rectify the broken relations. (left) Initially the seat-leg and seat-back attachments are broken. (middle) New edits, shifting the legs and scaling the back, are introduced to restore these broken relations. Consequently, the leg-bar and back-bar attachments are broken. This process continues until no relation remains broken, or all parts are edited.
• Figure 3: Searching for edits: Which edit operations could we use for the chair leg part? (a) First, broken constraints $\mathbb{C}$ (highlighted with a red line) are detected, denoting the relations to be fixed. (b) We then sample parameterizations of the edit operators in our DSL to create candidate edits $\mathbb{E}_R$. (c) Using CAS solvers, we search for $\textsc{Amount}$ expression for each edit candidate that satisfies the constraints $\mathbb{C}$, resulting in the set of valid edits $\mathbb{E}$.
• Figure 4: We compare parametric editing of 3D assets across three system variants. The One Shot LLM produces unrealistic part intersections (b, c) and fails to propagate corrective edits (a, d). The Ours Seed Only variant with Analytical Edit Propagation (AEP) produces consistent variations, but can fail to align with the input edit intent. Our full system (Ours), which includes edit type-hints and relation-validity, produces edits that closely match user requests.
• Figure 5: Proxydural Modeling: Leveraging the open-world knowledge of LLMs, we synthesize edit requests for a given shape to enable automatic procedural models, termed Proxydural due to the use of bounding proxy deformations. Our system allows multiple proxydural models for the same shape, a capability not possible with prior approaches bokeloh.