Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A reconfigurable smart camera implementation for jet flames characterization based on an optimized segmentation model

Gerardo Valente Vazquez-Garcia, Carmina Perez Guerrero, Eduardo Garduño, Miguel Gonzalez-Mendoza, Adriana Palacios, Gerardo Rodriguez-Hernandez, Vahid Foroughi, Alba Àgueda, Elsa Pastor, Gilberto Ochoa-Ruiz

Abstract

In this work we present a novel framework for fire safety management in industrial settings through the implementation of a smart camera platform for jet flames characterization. The approach seeks to alleviate the lack of real-time solutions for industrial early fire segmentation and characterization. As a case study, we demonstrate how a SoC FPGA, running optimized Artificial Intelligence (AI) models can be leveraged to implement a full edge processing pipeline for jet flames analysis. In this paper we extend previous work on computer-vision jet fire segmentation by creating a novel experimental set-up and system implementation for addressing this issue, which can be replicated to other fire safety applications. The proposed platform is designed to carry out image processing tasks in real-time and on device, reducing video processing overheads, and thus the overall latency. This is achieved by optimizing a UNet segmentation model to make it amenable for an SoC FPGAs implementation; the optimized model can then be efficiently mapped onto the SoC reconfigurable logic for massively parallel execution. For our experiments, we have chosen the Ultra96 platform, as it also provides the means for implementing full-fledged intelligent systems using the SoC peripherals, as well as other Operating System (OS) capabilities (i.e., multi-threading) for systems management. For optimizing the model we made use of the Vitis (Xilinx) framework, which enabled us to optimize the full precision model from 7.5 million parameters to 59,095 parameters (125x less), which translated into a reduction of the processing latency of 2.9x. Further optimization (multi-threading and batch normalization) led to an improvement of 7.5x in terms of latency, yielding a performance of 30 Frames Per Second (FPS) without sacrificing accuracy in terms of the evaluated metrics (Dice Score).

A reconfigurable smart camera implementation for jet flames characterization based on an optimized segmentation model

Abstract

In this work we present a novel framework for fire safety management in industrial settings through the implementation of a smart camera platform for jet flames characterization. The approach seeks to alleviate the lack of real-time solutions for industrial early fire segmentation and characterization. As a case study, we demonstrate how a SoC FPGA, running optimized Artificial Intelligence (AI) models can be leveraged to implement a full edge processing pipeline for jet flames analysis. In this paper we extend previous work on computer-vision jet fire segmentation by creating a novel experimental set-up and system implementation for addressing this issue, which can be replicated to other fire safety applications. The proposed platform is designed to carry out image processing tasks in real-time and on device, reducing video processing overheads, and thus the overall latency. This is achieved by optimizing a UNet segmentation model to make it amenable for an SoC FPGAs implementation; the optimized model can then be efficiently mapped onto the SoC reconfigurable logic for massively parallel execution. For our experiments, we have chosen the Ultra96 platform, as it also provides the means for implementing full-fledged intelligent systems using the SoC peripherals, as well as other Operating System (OS) capabilities (i.e., multi-threading) for systems management. For optimizing the model we made use of the Vitis (Xilinx) framework, which enabled us to optimize the full precision model from 7.5 million parameters to 59,095 parameters (125x less), which translated into a reduction of the processing latency of 2.9x. Further optimization (multi-threading and batch normalization) led to an improvement of 7.5x in terms of latency, yielding a performance of 30 Frames Per Second (FPS) without sacrificing accuracy in terms of the evaluated metrics (Dice Score).

Paper Structure

This paper contains 25 sections, 1 equation, 11 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Objectives
  3. State of the art
  4. Cloud vs edge
  5. Smart cameras
  6. Technological implementation choices
  7. Neural networks on smart cameras
  8. Smart cameras on the edge
  9. Methodology
  10. Case study
  11. Vitis AI software stack
  12. Full precision model
  13. model optimization
  14. Quantization of the network
  15. Pruning the network
  16. ...and 10 more sections

Figures (11)

  • Figure 1: Different types of flames patterns in industrial settings: pool, ball and jet fires. Modified from Palacios (2011).
  • Figure 2: Visual example of jet fire characterization based on segmentation of three temperature zones.
  • Figure 3: Overall flow followed in this work: a full precision model is trained using an dataset of jet flames. The model is then optimized using Vitis and the resulting moddel is ported on an -based architecture for evaluation (adapted from gardunno).
  • Figure 4: Edge computing vs cloud computing.
  • Figure 5: Comparison in terms of , flexibility, processing performance, power consumption and programmability of different technological choices for implementing smart cameras : a) ; b) Processor (i.e.,) and Digital Signal Processors ( ); c) and d) (figure adapted from Real2010 )
  • ...and 6 more figures