Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-grained spatial-temporal perception for gas leak segmentation

Xinlong Zhao, Shan Du

TL;DR

Gas leaks pose health and environmental risks, and segmentation in infrared video is challenging due to non-rigid, blurry shapes and scarce labeled data. The paper introduces FGSTP, an end-to-end framework that integrates a three-frame motion-aware stream with refined object features through a Consecutive Temporal Correlation (CTC) and Fine-grained Spatial Perception (FSP) modules, followed by a boundary-focused decoder. Training uses a hybrid loss $L_{hybrid}=L_{ce}^{w}+L_{iou}^{w}$ to balance pixel accuracy and shape alignment. On the GasVid dataset, FGSTP achieves state-of-the-art segmentation accuracy, validating the effectiveness of combining motion cues with fine-grained spatial perception and boundary refinement.

Abstract

Gas leaks pose significant risks to human health and the environment. Despite long-standing concerns, there are limited methods that can efficiently and accurately detect and segment leaks due to their concealed appearance and random shapes. In this paper, we propose a Fine-grained Spatial-Temporal Perception (FGSTP) algorithm for gas leak segmentation. FGSTP captures critical motion clues across frames and integrates them with refined object features in an end-to-end network. Specifically, we first construct a correlation volume to capture motion information between consecutive frames. Then, the fine-grained perception progressively refines the object-level features using previous outputs. Finally, a decoder is employed to optimize boundary segmentation. Because there is no highly precise labeled dataset for gas leak segmentation, we manually label a gas leak video dataset, GasVid. Experimental results on GasVid demonstrate that our model excels in segmenting non-rigid objects such as gas leaks, generating the most accurate mask compared to other state-of-the-art (SOTA) models.

Fine-grained spatial-temporal perception for gas leak segmentation

TL;DR

Gas leaks pose health and environmental risks, and segmentation in infrared video is challenging due to non-rigid, blurry shapes and scarce labeled data. The paper introduces FGSTP, an end-to-end framework that integrates a three-frame motion-aware stream with refined object features through a Consecutive Temporal Correlation (CTC) and Fine-grained Spatial Perception (FSP) modules, followed by a boundary-focused decoder. Training uses a hybrid loss to balance pixel accuracy and shape alignment. On the GasVid dataset, FGSTP achieves state-of-the-art segmentation accuracy, validating the effectiveness of combining motion cues with fine-grained spatial perception and boundary refinement.

Abstract

Gas leaks pose significant risks to human health and the environment. Despite long-standing concerns, there are limited methods that can efficiently and accurately detect and segment leaks due to their concealed appearance and random shapes. In this paper, we propose a Fine-grained Spatial-Temporal Perception (FGSTP) algorithm for gas leak segmentation. FGSTP captures critical motion clues across frames and integrates them with refined object features in an end-to-end network. Specifically, we first construct a correlation volume to capture motion information between consecutive frames. Then, the fine-grained perception progressively refines the object-level features using previous outputs. Finally, a decoder is employed to optimize boundary segmentation. Because there is no highly precise labeled dataset for gas leak segmentation, we manually label a gas leak video dataset, GasVid. Experimental results on GasVid demonstrate that our model excels in segmenting non-rigid objects such as gas leaks, generating the most accurate mask compared to other state-of-the-art (SOTA) models.
Paper Structure (11 sections, 5 equations, 4 figures, 2 tables)

This paper contains 11 sections, 5 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Framework Overview
  4. FGSTP Architecture
  5. Loss Function
  6. Experiments
  7. Dataset
  8. Metrics and Configurations
  9. Results
  10. Ablation Studies
  11. Conclusion

Figures (4)

  • Figure 1: The architecture of FGSTP. The model processes every current frame along with an adjacent frame each time. Each current frame needs to be processed twice with two adjacent frames. The encoder denoises the input frame and extracts the multi-scale features, while the decoder optimizes boundaries through the GRA and NCD modules. $f_{i}^j$ denotes the feature of the ith frame and jth scale. The Consecutive Temporal Correlation (CTC) block captures motion information. $f^j$ denotes the CTC output in the jth scale. The Fine-grained Spatial Perception (FSP) module refines spatial features.
  • Figure 2: The structure of CTC. It computes 4D correlation volume $Corr(f_t,f_{t-1})$ for motion information and integrates channel-wise semantics for precise motion alignment.
  • Figure 3: The structure of FSP. It enhances motion features from CTC by residual connection with encoder and calculates semantic representation.
  • Figure 4: Visualization results on GasVid Dataset. Our model prediction is the most accurate in different situations, $i.e.,$ close (camera) distance with complex background (cloud or birds interference, 1467), long distance with complex background (1476), medium distance with clear background (2559), long distance with clear background (2563), close distance with clear background (2566)