Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SynthRender and IRIS: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perception

Jose Moises Araya-Martinez, Thushar Tom, Adrián Sanchis Reig, Pablo Rey Valiente, Jens Lambrecht, Jörg Krüger

Abstract

Object perception is fundamental for tasks such as robotic material handling and quality inspection. However, modern supervised deep-learning perception models require large datasets for robust automation under semi-uncontrolled conditions. The cost of acquiring and annotating such data for proprietary parts is a major barrier for widespread deployment. In this context, we release SynthRender, an open source framework for synthetic image generation with Guided Domain Randomization capabilities. Furthermore, we benchmark recent Reality-to-Simulation techniques for 3D asset creation from 2D images of real parts. Combined with Domain Randomization, these synthetic assets provide low-overhead, transferable data even for parts lacking 3D files. We also introduce IRIS, the Industrial Real-Sim Imagery Set, containing 32 categories with diverse textures, intra-class variation, strong inter-class similarities and about 20,000 labels. Ablations on multiple benchmarks outline guidelines for efficient data generation with SynthRender. Our method surpasses existing approaches, achieving 99.1% mAP@50 on a public robotics dataset, 98.3% mAP@50 on an automotive benchmark, and 95.3% mAP@50 on IRIS.

SynthRender and IRIS: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perception

Abstract

Object perception is fundamental for tasks such as robotic material handling and quality inspection. However, modern supervised deep-learning perception models require large datasets for robust automation under semi-uncontrolled conditions. The cost of acquiring and annotating such data for proprietary parts is a major barrier for widespread deployment. In this context, we release SynthRender, an open source framework for synthetic image generation with Guided Domain Randomization capabilities. Furthermore, we benchmark recent Reality-to-Simulation techniques for 3D asset creation from 2D images of real parts. Combined with Domain Randomization, these synthetic assets provide low-overhead, transferable data even for parts lacking 3D files. We also introduce IRIS, the Industrial Real-Sim Imagery Set, containing 32 categories with diverse textures, intra-class variation, strong inter-class similarities and about 20,000 labels. Ablations on multiple benchmarks outline guidelines for efficient data generation with SynthRender. Our method surpasses existing approaches, achieving 99.1% mAP@50 on a public robotics dataset, 98.3% mAP@50 on an automotive benchmark, and 95.3% mAP@50 on IRIS.
Paper Structure (25 sections, 9 figures, 6 tables)

This paper contains 25 sections, 9 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Sim-to-real SynthRender Framework
  3. Automated DA Methods
  4. The IRIS
  5. Ablation Studies
  6. Related Works
  7. Domain Adaptation for Contextualized Synthetic Data
  8. Domain Randomization and the Simulation-to-Reality Gap
  9. Low-Overhead 3D Domain Adaptation
  10. Manual Modeling
  11. Manual CAD + MeshyAI
  12. 3DGS
  13. TRELLIS
  14. SynthRender Framework
  15. Load and process data
  16. ...and 10 more sections

Figures (9)

  • Figure 1: Architecture for efficient domain adaptation, domain randomization, and bidirectional sim-real gap reduction.
  • Figure 2: 3D asset and texture generation as DA approaches.
  • Figure 3: Functional diagram of SynthRender, illustrating the input, generation, and output stages.
  • Figure 4: IRIS objects. Synthetic renders of the 32 industrial CAD models used in this study.
  • Figure 5: SynthRender on three SOTA object detection models.
  • ...and 4 more figures