3D Shape Augmentation with Content-Aware Shape Resizing

TL;DR

The paper tackles data scarcity in 3D deep learning by introducing Efficient 3D Seam Carving (E3SC), a content-aware augmentation method that deforms only parts of a 3D model while preserving overall semantics. It adapts seam carving to 3D grids, using axis-specific energy $e_z(\boldsymbol{G}) = \left|\frac{\partial}{\partial z}\boldsymbol{G}\right|$ (and $e_3$) to guide seam paths, and employs rotation, beam search, and anchor points to generate multiple plausible augmentations within size bounds. Symmetry checks and diversity strategies (anchor clustering) ensure varied yet coherent outcomes, while performance trade-offs balance seam quality against computation time. Experiments on ShapeNetV2 demonstrate improvements in downstream 3D generation metrics (e.g., MMD, COV, 1-NNA) and favorable human preferences, particularly when training data are limited, highlighting E3SC’s practical impact for data-efficient 3D learning. The approach supports occupancy- and TSDF-based representations and can enhance tasks such as model generation, detection, and segmentation by expanding effective training sets.

Abstract

Recent advancements in deep learning for 3D models have propelled breakthroughs in generation, detection, and scene understanding. However, the effectiveness of these algorithms hinges on large training datasets. We address the challenge by introducing Efficient 3D Seam Carving (E3SC), a novel 3D model augmentation method based on seam carving, which progressively deforms only part of the input model while ensuring the overall semantics are unchanged. Experiments show that our approach is capable of producing diverse and high-quality augmented 3D shapes across various types and styles of input models, achieving considerable improvements over previous methods. Quantitative evaluations demonstrate that our method effectively enhances the novelty and quality of shapes generated by other subsequent 3D generation algorithms.

Figures (8)

• Figure 1: Given an input model (rendered in red in the middle), our approach has the capability to generate a variety of augmented 3D shapes characterized by intricate structures and precise details.
• Figure 2: Comparison between different energy functions. Given the same input model (a), we show three augmented results (b-d) using Eq. \ref{['eqn:e_z']}, and another three results (e-g) using Eq. \ref{['eqn:e_full']}. The anchors are not limited to occupied cells in both cases.
• Figure 3: The selection of the anchor point can notably impact the trajectory of the seam. Taking an occupancy grid of a plane as an example (a), the energy maps are reduced along the y-axis (b-g). The anchors (illustrated as red dots) vary along both the z-axis (b-d) and x-axis (e-g), and the cutting axis also transitions from the z-axis (b-d) to the x-axis (e-g). Both modifications contribute to a substantial divergence in the seams' paths.
• Figure 4: An illustration of seams computed using our algorithm, with different cutting axis employed as input.
• Figure 5: Illustration of augmented models based on input 3D shapes represented by TSDF grids. The first row of each column demonstrates the input models.