Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthetic imagery for fuzzy object detection: A comparative study

Siavash H. Khajavi, Mehdi Moshtaghi, Dikai Yu, Zixuan Liu, Kary Främling, Jan Holmström

TL;DR

This research proposes and leverage an alternative method of generating and automatically annotating fully synthetic fire images based on 3D models for training an object detection model and illustrates that when synthetic imagery and real imagery is utilized in a mixed training set the resulting ML model outperforms models trained on real imagery as well as models trained on synthetic imagery for detection of a broad spectrum of fires.

Abstract

The fuzzy object detection is a challenging field of research in computer vision (CV). Distinguishing between fuzzy and non-fuzzy object detection in CV is important. Fuzzy objects such as fire, smoke, mist, and steam present significantly greater complexities in terms of visual features, blurred edges, varying shapes, opacity, and volume compared to non-fuzzy objects such as trees and cars. Collection of a balanced and diverse dataset and accurate annotation is crucial to achieve better ML models for fuzzy objects, however, the task of collection and annotation is still highly manual. In this research, we propose and leverage an alternative method of generating and automatically annotating fully synthetic fire images based on 3D models for training an object detection model. Moreover, the performance, and efficiency of the trained ML models on synthetic images is compared with ML models trained on real imagery and mixed imagery. Findings proved the effectiveness of the synthetic data for fire detection, while the performance improves as the test dataset covers a broader spectrum of real fires. Our findings illustrates that when synthetic imagery and real imagery is utilized in a mixed training set the resulting ML model outperforms models trained on real imagery as well as models trained on synthetic imagery for detection of a broad spectrum of fires. The proposed method for automating the annotation of synthetic fuzzy objects imagery carries substantial implications for reducing both time and cost in creating computer vision models specifically tailored for detecting fuzzy objects.

Synthetic imagery for fuzzy object detection: A comparative study

TL;DR

This research proposes and leverage an alternative method of generating and automatically annotating fully synthetic fire images based on 3D models for training an object detection model and illustrates that when synthetic imagery and real imagery is utilized in a mixed training set the resulting ML model outperforms models trained on real imagery as well as models trained on synthetic imagery for detection of a broad spectrum of fires.

Abstract

The fuzzy object detection is a challenging field of research in computer vision (CV). Distinguishing between fuzzy and non-fuzzy object detection in CV is important. Fuzzy objects such as fire, smoke, mist, and steam present significantly greater complexities in terms of visual features, blurred edges, varying shapes, opacity, and volume compared to non-fuzzy objects such as trees and cars. Collection of a balanced and diverse dataset and accurate annotation is crucial to achieve better ML models for fuzzy objects, however, the task of collection and annotation is still highly manual. In this research, we propose and leverage an alternative method of generating and automatically annotating fully synthetic fire images based on 3D models for training an object detection model. Moreover, the performance, and efficiency of the trained ML models on synthetic images is compared with ML models trained on real imagery and mixed imagery. Findings proved the effectiveness of the synthetic data for fire detection, while the performance improves as the test dataset covers a broader spectrum of real fires. Our findings illustrates that when synthetic imagery and real imagery is utilized in a mixed training set the resulting ML model outperforms models trained on real imagery as well as models trained on synthetic imagery for detection of a broad spectrum of fires. The proposed method for automating the annotation of synthetic fuzzy objects imagery carries substantial implications for reducing both time and cost in creating computer vision models specifically tailored for detecting fuzzy objects.
Paper Structure (18 sections, 1 equation, 5 figures, 2 tables)

This paper contains 18 sections, 1 equation, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Iterations of synthetic data generation method
  5. Method 1: Process establishment
  6. Setting up virtual scenes using 3D models
  7. Defining camera movement rules
  8. Generating and arranging flame/fire image planes
  9. Calculating annotations
  10. Method 2: Process improvements
  11. Arranging flame/fire image planes to improve annotations
  12. Diversification of the scenes and fires
  13. ML experiments
  14. Experimental setup
  15. Comparison of synthetic data generation methods
  16. ...and 3 more sections

Figures (5)

  • Figure 1: The Method 2 flowchart for generating and automatically annotating the fully synthetic fire imagery
  • Figure 2: Comparison of background layers, flame object layers, annotation layers, and their relation to camera position used in Method 1 and Method 2. Method 1 (left): Random fire image plane overlay. Method 2 (right): Parallel with background overlay of fire image plane.
  • Figure 3: Comparison of annotation bounding box accuracy between 3D flame plane placement in Method 1 (left) and 2D layer flame generation in Method 2 (right)
  • Figure 4: Samples of SynthData 2 (synthetic images generated and annotated in Method 2) used in Experiment 2
  • Figure 5: Comparison of training combinations tested on (a) $\mathbf{FiIW_{test}}$ and (b) $\mathbf{RealRareFire}$