Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boosting AI Reliability with an FSM-Driven Streaming Inference Pipeline: An Industrial Case

Yutian Zhang, Zhongyi Pei, Yi Mao, Chen Wang, Lin Liu, Jianmin Wang

TL;DR

In experiments on a real-world dataset of over 7,000 images from 12 site videos, encompassing more than 300 excavator workloads, the method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules.

Abstract

The widespread adoption of AI in industry is often hampered by its limited robustness when faced with scenarios absent from training data, leading to prediction bias and vulnerabilities. To address this, we propose a novel streaming inference pipeline that enhances data-driven models by explicitly incorporating prior knowledge. This paper presents the work on an industrial AI application that automatically counts excavator workloads from surveillance videos. Our approach integrates an object detection model with a Finite State Machine (FSM), which encodes knowledge of operational scenarios to guide and correct the AI's predictions on streaming data. In experiments on a real-world dataset of over 7,000 images from 12 site videos, encompassing more than 300 excavator workloads, our method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules. We will release the code at https://github.com/thulab/video-streamling-inference-pipeline.

Boosting AI Reliability with an FSM-Driven Streaming Inference Pipeline: An Industrial Case

TL;DR

In experiments on a real-world dataset of over 7,000 images from 12 site videos, encompassing more than 300 excavator workloads, the method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules.

Abstract

The widespread adoption of AI in industry is often hampered by its limited robustness when faced with scenarios absent from training data, leading to prediction bias and vulnerabilities. To address this, we propose a novel streaming inference pipeline that enhances data-driven models by explicitly incorporating prior knowledge. This paper presents the work on an industrial AI application that automatically counts excavator workloads from surveillance videos. Our approach integrates an object detection model with a Finite State Machine (FSM), which encodes knowledge of operational scenarios to guide and correct the AI's predictions on streaming data. In experiments on a real-world dataset of over 7,000 images from 12 site videos, encompassing more than 300 excavator workloads, our method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules. We will release the code at https://github.com/thulab/video-streamling-inference-pipeline.
Paper Structure (21 sections, 5 figures, 1 table)

This paper contains 21 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Approach
  3. Event Identification by Object Detection
  4. State Transfer By Business State Machine
  5. Business States
  6. State Transfer and Event Rectification
  7. Data Engineering
  8. Dataset Construction.
  9. Annotation and Quality Control
  10. Preprocessing Pipeline
  11. Data Splitting and Leakage Prevention
  12. Engineering Trade-offs
  13. EVALUATION
  14. RQ1. Overall Performance
  15. Settings
  16. ...and 6 more sections

Figures (5)

  • Figure 1: Wrong detection results lead to counting fake workloads, which often occur due to unseen objects, non-target vehicles, and occlusions not covered in the training set.
  • Figure 2: The overview of our approach
  • Figure 3: The business states of excavator monitoring
  • Figure 4: The business states of excavator monitoring
  • Figure 5: Comparison of mistakes under different business logic.