Fully Procedural Synthetic Data from Simple Rules for Multi-View Stereo

Abstract

In this paper, we explore the design space of procedural rules for multi-view stereo (MVS). We demonstrate that we can generate effective training data using SimpleProc: a new, fully procedural generator driven by a very small set of rules using Non-Uniform Rational Basis Splines (NURBS), as well as basic displacement and texture patterns. At a modest scale of 8,000 images, our approach achieves superior results compared to manually curated images (at the same scale) sourced from games and real-world objects. When scaled to 352,000 images, our method yields performance comparable to--and in several benchmarks, exceeding--models trained on over 692,000 manually curated images. The source code and the data are available at https://github.com/princeton-vl/SimpleProc.

Figures (6)

• Figure 1: Fully procedural synthetic data from simple rules (top) is as effective as curated data from artists or 3D scans (bottom) for training multi-view stereo models.
• Figure 2: Procedural Data Generation Pipeline. Stage 1 generates NURBS surfaces from the lofting operation. Stage 2 applies displacements, textures, and material properties. Stage 3 arranges cameras, objects, lighting, and optional room boxes.
• Figure 3: Shape generation pipeline. Top: Profile curves (Starfish and Reptile styles) with control points in red (only one of the profiles is shown for each shape). Middle: Stem curves from 3D random walks. Bottom: Resulting lofted NURBS surfaces showing diverse smooth and sharp features.
• Figure 4: Shapes with Displacement and Textures. The top row shows displacements applied to base shapes. The middle row shows textures from brick, wave, or Perlin noise pattern. The bottom row shows complex textures created by boolean operations.
• Figure 5: Gallery of 10 random examples of our generated scenes