ArchComplete: Autoregressive 3D Architectural Design Generation with Hierarchical Diffusion-Based Upsampling

TL;DR

ArchComplete tackles the challenge of producing high-resolution, architecturally valid 3D models by coupling a discrete, patch-based 3D VQGAN with an autoregressive transformer for coarse geometry and a cascade of 3D conditional diffusion models for hierarchical upsampling. The 2.5D perceptual loss and PatchGAN discriminator jointly enforce global coherence and local fidelity, while a four-level diffusion hierarchy enables up to $8\times$ refinement to $512^{3}$ at voxel sizes near $9\text{cm}$. A new 3D house dataset supports 1:1 scale interiors and exteriors, enabling shape interpolation, unconditional synthesis, and plan-drawing/shape completions. Quantitative and qualitative results show superiority over baselines in quality, diversity, and memory efficiency, suggesting practical impact for architectural ideation and detailisation workflows.

Abstract

Recent advances in 3D generative models have shown promising results but often fall short in capturing the complexity of architectural geometries and topologies and fine geometric details at high resolutions. To tackle this, we present ArchComplete, a two-stage voxel-based 3D generative pipeline consisting of a vector-quantised model, whose composition is modelled with an autoregressive transformer for generating coarse shapes, followed by a hierarchical upsampling strategy for further enrichment with fine structures and details. Key to our pipeline is (i) learning a contextually rich codebook of local patch embeddings, optimised alongside a 2.5D perceptual loss that captures global spatial correspondence of projections onto three axis-aligned orthogonal planes, and (ii) redefining upsampling as a set of conditional diffusion models learning from a hierarchy of randomly cropped coarse-to-fine local volumetric patches. Trained on our introduced dataset of 3D house models with fully modelled exterior and interior, ArchComplete autoregressively generates models at the resolution of $64^{3}$ and progressively refines them up to $512^{3}$, with voxel sizes as small as $ \approx 9\text{cm}$. ArchComplete solves a variety of tasks, including genetic interpolation and variation, unconditional synthesis, shape and plan-drawing completion, as well as geometric detailisation, while achieving state-of-the-art performance in quality, diversity, and computational efficiency.

Figures (11)

• Figure 1: Example 3D houses of the dataset. The geometry is sliced with two cutting planes to reveal its corresponding interiority. Individual colours are representative of different materials.
• Figure 2: Pipeline Overview. The framework follows a two-stage 3D generative pipeline. In stage 1 (left), a 3D Voxel VQGAN is trained on dense voxelised data with a patch-wise quantisation step together with a 3D PatchGAN discriminator that enforces fidelity at the scale of local patches and a 2.5D perceptual loss to retain global spatial coherence. Followed by an autoregressive transformer architecture trained to synthesise new coarse voxelised models. In stage 2 (right), a set of 3D c-DDPMs models are trained over a hierarchy of local volumetric patches conditioned on coarser levels in a cascaded fashion. During inference, the coarse output is unfolded into overlapping patches, upsampled in parallel, and folded back into a high-resolution grid. This process continues until the highest desired resolution is achieved. Note that we achieve an $8X$ upsampling rate in our implementation, while the figure only depicts up to $4X$.
• Figure 3: Sampling chunks on a 3D house model and the corresponding voxelised patches in four different resolutions forming the ground-truth data for training hierarchical upsampling networks. Note that not all $100$ points are shown for visual ease.
• Figure 4: Three exemplars showcasing noisy voxel configuration. We detect such cases by computing the axis-wise $V_{c_{i}}$ and checking for their individual values and also their summation. (a) One acceptable (in green) and one noisy voxel, while (b) is a noisy sticking, and (c) is noisy floating voxel.
• Figure 5: Four interpolated house models generated by applying crossover operations on their corresponding token sequences.