TopoDiT-3D: Topology-Aware Diffusion Transformer with Bottleneck Structure for 3D Point Cloud Generation

TL;DR

TopoDiT-3D introduces a topology-aware diffusion Transformer for 3D point cloud generation, addressing token redundancy and the lack of global topology by integrating persistent homology via persistence images. It employs a bottleneck structure with a Perceiver Resampler to compress tokens and fuse local patch information with global topological cues, enabling efficient training and scalable generation. Across ShapeNet chair, airplane, and car categories, TopoDiT-3D achieves state-of-the-art visual quality and diversity while reducing training costs relative to prior diffusion-transformer baselines. The work demonstrates the value of incorporating rich topological information and its synergy with traditional local feature learning for robust 3D generation.

Abstract

Recent advancements in Diffusion Transformer (DiT) models have significantly improved 3D point cloud generation. However, existing methods primarily focus on local feature extraction while overlooking global topological information, such as voids, which are crucial for maintaining shape consistency and capturing complex geometries. To address this limitation, we propose TopoDiT-3D, a Topology-Aware Diffusion Transformer with a bottleneck structure for 3D point cloud generation. Specifically, we design the bottleneck structure utilizing Perceiver Resampler, which not only offers a mode to integrate topological information extracted through persistent homology into feature learning, but also adaptively filters out redundant local features to improve training efficiency. Experimental results demonstrate that TopoDiT-3D outperforms state-of-the-art models in visual quality, diversity, and training efficiency. Furthermore, TopoDiT-3D demonstrates the importance of rich topological information for 3D point cloud generation and its synergy with conventional local feature learning. Videos and code are available at https://github.com/Zechao-Guan/TopoDiT-3D.

Figures (23)

• Figure 1: Qualitative visualizations demonstrating high-fidelity and diverse 3D point cloud generation.
• Figure 2: Comparison of training efficiency between TopoDiT-3D and DiT-3D. Experiments were conducted with a batch size of 64, a voxel resolution of 32, and a patch size of 4 over 10,000 training epochs.
• Figure 3: The illustration demonstrates the filtration of Vietoris–Rips (VR) complexes, along with the corresponding persistence barcodes, diagrams, and images. As the radius $r$ increases, simplexes are added, leading to the birth and death of topological characteristics. Specifically: (a) Persistence barcodes record the lifespan of homological invariants, with 0-dimension (connected components) in orange and 1-dimension (loops) in brown. (b) Persistence diagram maps birth-death pairs as points, using the same color scheme. (c) and (d) apply a transformation $T$ to map these pairs onto a grid, generating persistence images that encode 0D and 1D topological information.
• Figure 4: The illustration of the proposed Topology-Aware Diffusion Transformer (TopoDiT-3D) for 3D point cloud generation. TopoDiT-3D initially voxelizes the point clouds, employing the patch operator to generate tokens related to the local point-voxel feature, and the persistence images to generate tokens related to the global topological feature. The persistence images are generated by the pretrained VAE during inference. Subsequently, TopoDiT-3D uses a fixed minor number of learned queries and the Perceiver Resampler to downsample and learn the topological and geometric information. After $N$ DiT blocks, it uses the Perceiver Resampler to achieve upsampling, which recovers the same number of patch tokens to devoxelize.
• Figure 5: The Perceiver Resampler module.

Theorems & Definitions (2)

• Definition 1
• Definition 2