Visual Deformation Detection Using Soft Material Simulation for Pre-training of Condition Assessment Models
Joel Sol, Amir M. Soufi Enayati, Homayoun Najjaran
TL;DR
The paper tackles the problem of deformity detection for geometric quality assurance by learning from synthetic data generated in Blender. It presents a pipeline that uses shape-key driven deformations, randomized viewpoints, and background augmentation to create labeled images of deformed and non-deformed objects, then pre-trains a convolutional network on this data. The approach demonstrates that synthetic data can support sim-to-real transfer, with background diversity (BG-20k) reducing the gap to real-world performance. The work highlights practical implications for reducing reliance on costly real data and suggests future directions in domain adaptation through GANs and transfer learning to further close the sim-to-real gap.
Abstract
This paper addresses the challenge of geometric quality assurance in manufacturing, particularly when human assessment is required. It proposes using Blender, an open-source simulation tool, to create synthetic datasets for machine learning (ML) models. The process involves translating expert information into shape key parameters to simulate deformations, generating images for both deformed and non-deformed objects. The study explores the impact of discrepancies between real and simulated environments on ML model performance and investigates the effect of different simulation backgrounds on model sensitivity. Additionally, the study aims to enhance the model's robustness to camera positioning by generating datasets with a variety of randomized viewpoints. The entire process, from data synthesis to model training and testing, is implemented using a Python API interfacing with Blender. An experiment with a soda can object validates the accuracy of the proposed pipeline.