SYNOSIS: Image synthesis pipeline for machine vision in metal surface inspection

TL;DR

This work presents the first synthetic data generation pipeline that is capable of generating large datasets of physically realistic textures exhibiting sophisticated structured patterns, and publishes the dual dataset used in this work, presenting models of sandblasting, parallel, and spiral milling textures.

Abstract

The use of machine learning (ML) methods for development of robust and flexible visual inspection system has shown promising. However their performance is highly dependent on the amount and diversity of training data. This is often restricted not only due to costs but also due to a wide variety of defects and product surfaces which occur with varying frequency. As such, one can not guarantee that the acquired dataset contains enough defect and product surface occurrences which are needed to develop a robust model. Using parametric synthetic dataset generation, it is possible to avoid these issues. In this work, we introduce a complete pipeline which describes in detail how to approach image synthesis for surface inspection - from first acquisition, to texture and defect modeling, data generation, comparison to real data and finally use of the synthetic data to train a defect segmentation model. The pipeline is in detail evaluated for milled and sandblasted aluminum surfaces. In addition to providing an in-depth view into each step, discussion of chosen methods, and presentation of ML results, we provide a comprehensive dual dataset containing both real and synthetic images.

Figures (21)

• Figure 1: SYNOSIS synthesis pipeline. Based on a 3D model of an object, spatial properties of the surface texture are measured. Surface topography and manufacturing parameters are used to develop a mathematical model capable of reproducing the texture as a normal map. Multiple realizations of the texture are generated by varying the parameters and are applied onto the 3D model of the object which has been altered to include surface defects of varying types and sizes. During the simulation step, the final image is computed based on the interaction between the acquisition environment (light, camera), object geometry and texture normal map. This process is repeated an arbitrary amount of times. Parameters defining texture and surface defects are varied to generate a training dataset which is sufficient in terms of both image quantity and content variance.
• Figure 2: Examples of the three textures considered in this paper: surface finishing by sandblasting, parallel milling, and spiral milling.
• Figure 3: Image synthesis overview. Texture and defect information is joined with the 3D scene to perform rendering and generate an image. The defect information is varied to perform photo-realistic image synthesis of both defected and defect-free object instances. In case of defected instances, pixel-precise defect annotations are automatically created.
• Figure 4: Simulated 3D scene representing the real inspection environment.
• Figure 5: Real images of object surface with defects. Top row: 20 degrees angle from perpendicular view. Bottom row: 40 degrees angle from perpendicular view.