SynRailObs: A Synthetic Dataset for Obstacle Detection in Railway Scenarios

TL;DR

SynRailObs tackles data scarcity and distribution shift in railway obstacle detection by introducing a high-fidelity synthetic dataset generated from diverse background rail scenes and diffusion-generated foreground obstacles. The pipeline leverages SAM for obstacle extraction, perspective-aware pasting, and image harmony to produce realistic, annotation-free images, further enriched by zero-shot obstacle generation. Real-world evaluations across ballastless and ballasted tracks under multiple weather conditions show robust detection performance and strong zero-shot capabilities, with some degradation in extreme conditions due to limited adverse-weather backgrounds. The dataset is publicly available on Kaggle, enabling broader research and practical training of obstacle detectors for railway safety across distances and environments. Overall, SynRailObs demonstrates the potential of synthetic data to bridge gaps in real-world railway safety benchmarks and support safer high-speed railway operations.

Abstract

Detecting potential obstacles in railway environments is critical for preventing serious accidents. Identifying a broad range of obstacle categories under complex conditions requires large-scale datasets with precisely annotated, high-quality images. However, existing publicly available datasets fail to meet these requirements, thereby hindering progress in railway safety research. To address this gap, we introduce SynRailObs, a high-fidelity synthetic dataset designed to represent a diverse range of weather conditions and geographical features. Furthermore, diffusion models are employed to generate rare and difficult-to-capture obstacles that are typically challenging to obtain in real-world scenarios. To evaluate the effectiveness of SynRailObs, we perform experiments in real-world railway environments, testing on both ballasted and ballastless tracks across various weather conditions. The results demonstrate that SynRailObs holds substantial potential for advancing obstacle detection in railway safety applications. Models trained on this dataset show consistent performance across different distances and environmental conditions. Moreover, the model trained on SynRailObs exhibits zero-shot capabilities, which are essential for applications in security-sensitive domains. The data is available in https://www.kaggle.com/datasets/qiushi910/synrailobs.

Figures (4)

• Figure 1: Sample images in SynRailObs. From top to bottom, left to right are persons, animals, motorcycle, rocks, vehicles and random polygons.
• Figure 2: Workflow of synthetic image generation. In obstacle brach, Potential obstacles are extracted from stable-diffusion generated images and public datasets guided by SAM. The extracted obstacles form an obstacle gallery; In background branch, railway area are determined by SAM leveraging point prompts. The obstacles are pasted on railway area after rescale and harmonization.
• Figure 3: The first and second columns represent the prediction confidences of the YOLO model pretrained on Objects365 for unharmonized and harmonized images, respectively.The third column shows the Grad-CAM visualizations of the corresponding regions.
• Figure 4: Samples of test images captured in our experimental site.