SmartRSD: An Intelligent Multimodal Approach to Real-Time Road Surface Detection for Safe Driving

TL;DR

The effectiveness and reliability of the proposed multimodal approach in accurately identifying road surface conditions in real-time scenarios are demonstrated and highlight the potential of integrating auditory and visual cues for enhancing road safety and minimizing accident risks.

Abstract

Precise and prompt identification of road surface conditions enables vehicles to adjust their actions, like changing speed or using specific traction control techniques, to lower the chance of accidents and potential danger to drivers and pedestrians. However, most of the existing methods for detecting road surfaces solely rely on visual data, which may be insufficient in certain situations, such as when the roads are covered by debris, in low light conditions, or in the presence of fog. Therefore, we introduce a multimodal approach for the automated detection of road surface conditions by integrating audio and images. The robustness of the proposed method is tested on a diverse dataset collected under various environmental conditions and road surface types. Through extensive evaluation, we demonstrate the effectiveness and reliability of our multimodal approach in accurately identifying road surface conditions in real-time scenarios. Our findings highlight the potential of integrating auditory and visual cues for enhancing road safety and minimizing accident risks

Figures (4)

• Figure 1: Overview of the Proposed Multimodal Fusion
• Figure 2: Overview of the Proposed (a) Improved MobileNet, and (b) Improved YAMNet architecture for SmartRSD
• Figure 3: Raw Audio
• Figure 4: Spectogram