Generate Point Clouds with Multiscale Details from Graph-Represented Structures

TL;DR

This work addresses the challenge of controllable detail generation in 3D point clouds by introducing the Multiscale Structure Graph (MSG), a graph representation that captures structure across multiple scales. It couples MSG with the Multiscale Structure-based Point Cloud Generator (MSPCG), which learns from local multi-scale patterns using a similarity-transform invariant pipeline built on a relative Graph Attention Network and per-vertex canonicalization. A weighted Chamfer loss balances contributions from vertices of different capacities, and the framework supports editing and scene-level generation by adjusting the MSG. Experiments on ShapeNet, ModelNet, and ScanNet demonstrate improved generalization across categories and robust reconstruction and multiscale editing capabilities, highlighting practical potential for scalable, detail-rich 3D generation. The method advances controllable point cloud synthesis by leveraging cross-scale pattern reuse and invariant representations, enabling reliable generation and editing in diverse domains.

Abstract

As details are missing in most representations of structures, the lack of controllability to more information is one of the major weaknesses in structure-based controllable point cloud generation. It is observable that definitions of details and structures are subjective. Details can be treated as structures on small scales. To represent structures in different scales at the same time, we present a graph-based representation of structures called the Multiscale Structure Graph (MSG). Given structures in multiple scales, similar patterns of local structures can be found at different scales, positions, and angles. The knowledge learned from a regional structure pattern shall be transferred to other similar patterns. An encoding and generation mechanism, namely the Multiscale Structure-based Point Cloud Generator (MSPCG) is proposed, which can simultaneously learn point cloud generation from local patterns with miscellaneous spatial properties. The proposed method supports multiscale editions on point clouds by editing the MSG. By generating point clouds from local structures and learning simultaneously in multiple scales, our MSPCG has better generalization ability and scalability. Trained on the ShapeNet, our MSPCG can generate point clouds from a given structure for unseen categories and indoor scenes. The experimental results show that our method significantly outperforms baseline methods.

Figures (6)

• Figure 1: (a) An example of a Multiscale Structure Graph. Vertices with larger capacity imply more points in the generated point cloud and vice versa. (b) Examples of similar local structure patterns can help generate similar patterns in other categories and scales.
• Figure 2: Overview framework of proposed MSPCG. K is the number of vertices and N is the number of points we finally generate. $^*$Relative Capacity ratios are calculated in 3-ring neighbors because the depth of GAT is 3.
• Figure 3: Reconstruction Chamfer Distances in ShapeNet Unseen when all point clouds are scaled in a certain ratio.
• Figure 4: Reconstruction results on ShapeNet and ModelNet. Note that our MSPCG takes structure graphs as input and other methods use point clouds.
• Figure 5: Examples of multiscale editing.