RoadFed: A Multimodal Federated Learning System for Improving Road Safety

TL;DR

RoadFed proposes a privacy-preserving, multimodal federated learning system for road hazard detection in ITS. It introduces MRHD for cross-modal hazard recognition, MFed to cut communication while handling non-iid data, and MLDP to guard privacy in high-dimensional multimodal inputs. Empirical results show superior accuracy (up to 96.42% overall on real-world data), very low latency (0.0351s), and substantially reduced communication costs compared to existing methods, with hazard results visualized on Google Maps. The framework demonstrates robust edge-cloud collaboration, privacy guarantees, and practical deployment potential for cooperative intelligent transportation systems.

Abstract

Internet of Things (IoTs) have been widely applied in Collaborative Intelligent Transportation Systems (C-ITS) for the prevention of road accidents. As one of the primary causes of road accidents in C-ITS, the efficient detection and early alarm of road hazards are of paramount importance. Given the importance, extensive research has explored this topic and obtained favorable results. However, most existing solutions only explore single-modality data, struggle with high computation and communication overhead, or suffer from the curse of high dimensionality in their privacy-preserving methodologies. To overcome these obstacles, in this paper, we introduce RoadFed, an innovative and private multimodal Federated learning-based system tailored for intelligent Road hazard detection and alarm. This framework encompasses an innovative Multimodal Road Hazard Detector, a communication-efficient federated learning approach, and a customized low-error-rate local differential privacy method crafted for high dimensional multimodal data. Experimental results reveal that the proposed RoadFed surpasses most existing systems in the self-gathered real-world and CrisisMMD public datasets. In particular, RoadFed achieves an accuracy of 96.42% with a mere 0.0351 seconds of latency and its communication cost is up to 1,000 times lower than existing systems in this field. It facilitates collaborative training with non-iid high dimensional multimodal real-world data across various data modalities on multiple edges while ensuring privacy preservation for road users.

Figures (10)

• Figure 1:
• Figure 2: The proposed Multimodal Road Hazard Detector utilizes a triplet loss to improve feature quality, i.e., enlarging inter-class features’ distances and shrinking intra-class features’ distances, for higher accuracy.
• Figure 3: Example images and texts of road hazards (from left to right, crashed vehicles, icy road, fallen tree, and damaged road), including dangerous type and location.
• Figure 4: MRHD's accuracy over different $\alpha$.
• Figure 5: Performance comparison of MFed on both the road danger dataset and the MNIST public dataset (MFed-Q and MFed-LRD refer to MFed without using model quantization and learning rate decay mechanism, respectively).