Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comprehensive Machine Learning Benchmarking for Fringe Projection Profilometry with Photorealistic Synthetic Data

Anush Lakshman S, Adam Haroon, Beiwen Li

TL;DR

The paper tackles the lack of large-scale ground-truth data and benchmarking for fringe projection profilometry (FPP) by introducing VIRTUS-FPP, a photorealistic synthetic FPP framework in NVIDIA Isaac Sim. It provides the first open-source dataset consisting of 15,600 fringe images and 300 depth maps for 50 objects, along with standardized evaluation protocols. A benchmarking study across four architectures (UNet, Hformer, ResUNet, Pix2Pix) reveals nearly identical single-shot RMSE (approximately $58$–$60$ mm) with notable geometry-dependent variance and a clear underperformance of ResUNet, underscoring fundamental limitations of end-to-end fringe-to-depth regression without explicit phase information. The findings motivate hybrid approaches that integrate phase information and multi-view cues, and the work lays a foundation for sim-to-real transfer and domain adaptation to advance robust FPP systems for manufacturing and inspection.

Abstract

Machine learning approaches for fringe projection profilometry (FPP) are hindered by the lack of large, diverse datasets and comprehensive benchmarking protocols. This paper introduces the first open-source, photorealistic synthetic dataset for FPP, generated using NVIDIA Isaac Sim with 15,600 fringe images and 300 depth reconstructions across 50 diverse objects. We benchmark four neural network architectures (UNet, Hformer, ResUNet, Pix2Pix) on single-shot depth reconstruction, revealing that all models achieve similar performance (58-77 mm RMSE) despite substantial architectural differences. Our results demonstrate fundamental limitations of direct fringe-to-depth mapping without explicit phase information, with reconstruction errors approaching 75-95\% of the typical object depth range. This resource provides standardized evaluation protocols enabling systematic comparison and development of learning-based FPP approaches.

Comprehensive Machine Learning Benchmarking for Fringe Projection Profilometry with Photorealistic Synthetic Data

TL;DR

The paper tackles the lack of large-scale ground-truth data and benchmarking for fringe projection profilometry (FPP) by introducing VIRTUS-FPP, a photorealistic synthetic FPP framework in NVIDIA Isaac Sim. It provides the first open-source dataset consisting of 15,600 fringe images and 300 depth maps for 50 objects, along with standardized evaluation protocols. A benchmarking study across four architectures (UNet, Hformer, ResUNet, Pix2Pix) reveals nearly identical single-shot RMSE (approximately – mm) with notable geometry-dependent variance and a clear underperformance of ResUNet, underscoring fundamental limitations of end-to-end fringe-to-depth regression without explicit phase information. The findings motivate hybrid approaches that integrate phase information and multi-view cues, and the work lays a foundation for sim-to-real transfer and domain adaptation to advance robust FPP systems for manufacturing and inspection.

Abstract

Machine learning approaches for fringe projection profilometry (FPP) are hindered by the lack of large, diverse datasets and comprehensive benchmarking protocols. This paper introduces the first open-source, photorealistic synthetic dataset for FPP, generated using NVIDIA Isaac Sim with 15,600 fringe images and 300 depth reconstructions across 50 diverse objects. We benchmark four neural network architectures (UNet, Hformer, ResUNet, Pix2Pix) on single-shot depth reconstruction, revealing that all models achieve similar performance (58-77 mm RMSE) despite substantial architectural differences. Our results demonstrate fundamental limitations of direct fringe-to-depth mapping without explicit phase information, with reconstruction errors approaching 75-95\% of the typical object depth range. This resource provides standardized evaluation protocols enabling systematic comparison and development of learning-based FPP approaches.
Paper Structure (17 sections, 7 equations, 3 figures, 2 tables)

This paper contains 17 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction and Related Work
  2. Virtual Fringe Projection Profilometry
  3. System Configuration
  4. Virtual Calibration
  5. Data Acquisition Methodology
  6. Dataset Composition
  7. Fringe Acquisition and Ground Truth Generation
  8. Dataset Summary
  9. Single-Shot Reconstruction Benchmarking
  10. Problem Formulation
  11. Network Architectures
  12. Training and Evaluation
  13. Experimental Results
  14. Quantitative Comparison
  15. Qualitative Analysis
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Virtual camera–projector calibration setup with a pinhole camera model, rectangular light-source projector, calibration board, and matte background plane.
  • Figure 2: RMSE and MAE distributions and per-sample errors for all four models. Left: error distributions, center: RMSE/MAE distributions, right: per-sample errors with mean lines. Rows: UNet, Hformer, ResUNet, Pix2Pix. Error curves track closely for Pix2Pix, Hformer, and UNet.
  • Figure 3: Qualitative results for one test object: From left to right we have the ground truth normalized depth, prediction normalized depth, and absolute error. From top to bottom, the models are UNet, Hformer, ResUNet, and Pix2Pix. Models overall capture coarse shape but fail on fine details and accurate depth prediction.