Multimodal Learning for Arcing Detection in Pantograph-Catenary Systems

TL;DR

This work tackles the challenge of detecting transient arcing at the pantograph-catenary interface by fusing high-resolution visual data with contact-force measurements. It introduces MultiDeepSAD, a multimodal extension of DeepSAD that uses modality-specific encoders, pseudo-anomaly generation, and an exponential loss to robustly separate normal and anomalous states under data scarcity and domain shifts. Two datasets are created: SBB-AD from real SBB data and Open-AD derived from internet videos with synthetic force signals, enabling comprehensive evaluation. Results show that multimodal fusion significantly improves arcing detection (AUROC up to $95.03\%$ on Open-AD) and demonstrates resilience to distortions and limited anomaly labels, highlighting practical potential for maintenance optimization in rail systems. The work also provides publicly available benchmarks and outlines directions for real-time deployment and integration with spatial-temporal tracking for cumulative damage assessment.

Abstract

The pantograph-catenary interface is essential for ensuring uninterrupted and reliable power delivery in electrified rail systems. However, electrical arcing at this interface poses serious risks, including accelerated wear of contact components, degraded system performance, and potential service disruptions. Detecting arcing events at the pantograph-catenary interface is challenging due to their transient nature, noisy operating environment, data scarcity, and the difficulty of distinguishing arcs from other similar transient phenomena. To address these challenges, we propose a novel multimodal framework that combines high-resolution image data with force measurements to more accurately and robustly detect arcing events. First, we construct two arcing detection datasets comprising synchronized visual and force measurements. One dataset is built from data provided by the Swiss Federal Railways (SBB), and the other is derived from publicly available videos of arcing events in different railway systems and synthetic force data that mimic the characteristics observed in the real dataset. Leveraging these datasets, we propose MultiDeepSAD, an extension of the DeepSAD algorithm for multiple modalities with a new loss formulation. Additionally, we introduce tailored pseudo-anomaly generation techniques specific to each data type, such as synthetic arc-like artifacts in images and simulated force irregularities, to augment training data and improve the discriminative ability of the model. Through extensive experiments and ablation studies, we demonstrate that our framework significantly outperforms baseline approaches, exhibiting enhanced sensitivity to real arcing events even under domain shifts and limited availability of real arcing observations.

Figures (7)

• Figure 1: Proposed solution for arcing detection in pantograph-catenary systems with image and force measurements.
• Figure 2: Representative samples from the SBB-AD dataset, consisting of time-synchronized pantograph camera images and corresponding contact-force measurements provided by SBB.
• Figure 3: Examples from the Open-AD dataset, constructed from internet and simulation data.
• Figure 4: Illustration of the DeepSAD objective: normal samples are embedded close to a hypersphere center, while anomalous samples are pushed outside the hypersphere beyond a margin.
• Figure 5: Illustration of pseudo-anomaly image samples generation, where we randomly paste arcing images onto pantographs in normal images.