Train Ego-Path Detection on Railway Tracks Using End-to-End Deep Learning

TL;DR

The paper introduces train ego-path detection as a focused onboard-vision task and extends RailSem19 with ego-path annotations. It presents TEP-Net, a decoder-free regression model that predicts left/right ego-path rails at $H=64$ anchors plus a $y$-limit, trained with a domain-specific composite loss and online ROI augmentation. Empirical results show IoU around $0.975$ across backbones with sub-millisecond to a few-millisecond latency, and a clear trade-off where segmentation offers slightly higher accuracy at the cost of latency, while regression provides the best speed and competitive accuracy. By releasing code and the ego-path-annotated dataset, the work lays groundwork for robust, real-time ego-path perception that can underpin driver-assistance and autonomous rail operation systems, with future work aimed at incorporating temporal context.

Abstract

This paper introduces the task of "train ego-path detection", a refined approach to railway track detection designed for intelligent onboard vision systems. Whereas existing research lacks precision and often considers all tracks within the visual field uniformly, our proposed task specifically aims to identify the train's immediate path, or "ego-path", within potentially complex and dynamic railway environments. Building on this, we extend the RailSem19 dataset with ego-path annotations, facilitating further research in this direction. At the heart of our study lies TEP-Net, an end-to-end deep learning framework tailored for ego-path detection, featuring a configurable model architecture, a dynamic data augmentation strategy, and a domain-specific loss function. Leveraging a regression-based approach, TEP-Net outperforms SOTA: while addressing the track detection problem in a more nuanced way than previously, our model achieves 97.5% IoU on the test set and is faster than all existing methods. Further comparative analysis highlights the relevance of the conceptual choices behind TEP-Net, demonstrating its inherent propensity for robustness across diverse environmental conditions and operational dynamics. This work opens promising avenues for the development of intelligent driver assistance systems and autonomous train operations, paving the way toward safer and more efficient railway transportation.

Figures (12)

• Figure 1: Ego-perspective of a train approaching a switch, thus facing a diverging track. The train’s ego-path, accurately identified using the proposed method, is highlighted in green. The track portion in red, manually annotated for illustrative purposes, is unsafe for travel in this scenario.
• Figure 2: In this image, possible tracks are highlighted in blue. This detection area excludes non-forward-connected side tracks, thereby focusing on a more relevant ROI.
• Figure 3: Close-up view of a railway switch mechanism. The switch blades configuration is highlighted: closed on the left side and open on the right side. The bottom wheel-rail schematics illustrate the train's wheels, featuring a special profile with a flange, interacting with the switch blades. This clarifies why a train approaching this switch will be routed to the right, as confirmed by the directional arrow on the sign standing on the right of the image.
• Figure 4: Illustration of our data augmentation strategy. Green polylines delineate the ego-path ground truth, encapsulated by the yellow rectangle that defines the base ROI. The orange extension expands the ROI to place the base of the rails at its center, while the red rectangle adds predefined margins. To this resulting ROI, variability is introduced by applying stochastic shifts to the left, top, and right borders, conforming to a normal distribution with tunable parameters. This introduced variability is subject to constraints ensuring that the rail base remains entirely within the resulting images. Four examples of augmented versions of the top image are shown in the bottom row.
• Figure 5: TEP-Net architecture and prediction methodology: starting from a cropped and resized input image, features are extracted via the backbone, passed through dense layers for regressing the output values, which are then processed to produce the final ego-path prediction.