Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Universal Railway Obstacle Detection System based on Semi-supervised Segmentation And Optical Flow

Qiushi Guo

TL;DR

This work reformulates the task as a binary segmentation problem instead of the traditional object detection approach, and generates highly realistic synthetic images using Segment Anything (SAM) and YOLO, eliminating the need for manual annotation to produce abundant pixel-level annotations.

Abstract

Detecting obstacles in railway scenarios is both crucial and challenging due to the wide range of obstacle categories and varying ambient conditions such as weather and light. Given the impossibility of encompassing all obstacle categories during the training stage, we address this out-of-distribution (OOD) issue with a semi-supervised segmentation approach guided by optical flow clues. We reformulate the task as a binary segmentation problem instead of the traditional object detection approach. To mitigate data shortages, we generate highly realistic synthetic images using Segment Anything (SAM) and YOLO, eliminating the need for manual annotation to produce abundant pixel-level annotations. Additionally, we leverage optical flow as prior knowledge to train the model effectively. Several experiments are conducted, demonstrating the feasibility and effectiveness of our approach.

A Universal Railway Obstacle Detection System based on Semi-supervised Segmentation And Optical Flow

TL;DR

This work reformulates the task as a binary segmentation problem instead of the traditional object detection approach, and generates highly realistic synthetic images using Segment Anything (SAM) and YOLO, eliminating the need for manual annotation to produce abundant pixel-level annotations.

Abstract

Detecting obstacles in railway scenarios is both crucial and challenging due to the wide range of obstacle categories and varying ambient conditions such as weather and light. Given the impossibility of encompassing all obstacle categories during the training stage, we address this out-of-distribution (OOD) issue with a semi-supervised segmentation approach guided by optical flow clues. We reformulate the task as a binary segmentation problem instead of the traditional object detection approach. To mitigate data shortages, we generate highly realistic synthetic images using Segment Anything (SAM) and YOLO, eliminating the need for manual annotation to produce abundant pixel-level annotations. Additionally, we leverage optical flow as prior knowledge to train the model effectively. Several experiments are conducted, demonstrating the feasibility and effectiveness of our approach.

Paper Structure

This paper contains 13 sections, 8 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Obstacle Detection in Railyway
  4. Segmentation with Optical flow
  5. Method
  6. Data Acquisition
  7. Optical-Flow
  8. Experiments
  9. Dataset and Evaluation Metrics
  10. Implementation Details
  11. Results
  12. Ablation Study
  13. Conclusion

Figures (4)

  • Figure 1: The pipeline for synthetic data generation, utilizing SAM and YOLO to extract target objects from a gallery and superimpose them onto a base image (specifically, a railway image). Notably, this process does not necessitate annotations.
  • Figure 2: Pipeline of our proposed method.
  • Figure 3: Sample base images depicting various weather conditions. From left to right, the images illustrate scenes captured under foggy, normal, and rainy conditions.
  • Figure 4: Sample of generated image-mask pairs. From left to right: pedestrian, animal and texture