Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DSDFormer: An Innovative Transformer-Mamba Framework for Robust High-Precision Driver Distraction Identification

Junzhou Chen, Zirui Zhang, Jing Yu, Heqiang Huang, Ronghui Zhang, Xuemiao Xu, Bin Sheng, Hong Yan

TL;DR

DSDFormer introduces a Transformer-Mamba fusion via Dual State Domain Attention to capture both global context and fine-grained local cues for driver distraction detection, addressing noisy video annotations with Temporal Reasoning Confident Learning (TRCL). The architecture enriches features through the SCEM, MBEM, and LFFN modules and demonstrates state-of-the-art accuracy on AUC-V1, AUC-V2, and 100-Driver, while maintaining real-time inference on edge devices (22 FPS on Jetson AGX Orin). TRCL autonomously cleans noisy labels by leveraging spatiotemporal frame correlations, improving data quality and model resilience. Collectively, DSDFormer and TRCL offer a scalable, robust solution for reliable driver-distraction identification in intelligent transportation systems, with potential applicability to other real-time action-recognition tasks.

Abstract

Driver distraction remains a leading cause of traffic accidents, posing a critical threat to road safety globally. As intelligent transportation systems evolve, accurate and real-time identification of driver distraction has become essential. However, existing methods struggle to capture both global contextual and fine-grained local features while contending with noisy labels in training datasets. To address these challenges, we propose DSDFormer, a novel framework that integrates the strengths of Transformer and Mamba architectures through a Dual State Domain Attention (DSDA) mechanism, enabling a balance between long-range dependencies and detailed feature extraction for robust driver behavior recognition. Additionally, we introduce Temporal Reasoning Confident Learning (TRCL), an unsupervised approach that refines noisy labels by leveraging spatiotemporal correlations in video sequences. Our model achieves state-of-the-art performance on the AUC-V1, AUC-V2, and 100-Driver datasets and demonstrates real-time processing efficiency on the NVIDIA Jetson AGX Orin platform. Extensive experimental results confirm that DSDFormer and TRCL significantly improve both the accuracy and robustness of driver distraction detection, offering a scalable solution to enhance road safety.

DSDFormer: An Innovative Transformer-Mamba Framework for Robust High-Precision Driver Distraction Identification

TL;DR

DSDFormer introduces a Transformer-Mamba fusion via Dual State Domain Attention to capture both global context and fine-grained local cues for driver distraction detection, addressing noisy video annotations with Temporal Reasoning Confident Learning (TRCL). The architecture enriches features through the SCEM, MBEM, and LFFN modules and demonstrates state-of-the-art accuracy on AUC-V1, AUC-V2, and 100-Driver, while maintaining real-time inference on edge devices (22 FPS on Jetson AGX Orin). TRCL autonomously cleans noisy labels by leveraging spatiotemporal frame correlations, improving data quality and model resilience. Collectively, DSDFormer and TRCL offer a scalable, robust solution for reliable driver-distraction identification in intelligent transportation systems, with potential applicability to other real-time action-recognition tasks.

Abstract

Driver distraction remains a leading cause of traffic accidents, posing a critical threat to road safety globally. As intelligent transportation systems evolve, accurate and real-time identification of driver distraction has become essential. However, existing methods struggle to capture both global contextual and fine-grained local features while contending with noisy labels in training datasets. To address these challenges, we propose DSDFormer, a novel framework that integrates the strengths of Transformer and Mamba architectures through a Dual State Domain Attention (DSDA) mechanism, enabling a balance between long-range dependencies and detailed feature extraction for robust driver behavior recognition. Additionally, we introduce Temporal Reasoning Confident Learning (TRCL), an unsupervised approach that refines noisy labels by leveraging spatiotemporal correlations in video sequences. Our model achieves state-of-the-art performance on the AUC-V1, AUC-V2, and 100-Driver datasets and demonstrates real-time processing efficiency on the NVIDIA Jetson AGX Orin platform. Extensive experimental results confirm that DSDFormer and TRCL significantly improve both the accuracy and robustness of driver distraction detection, offering a scalable solution to enhance road safety.
Paper Structure (20 sections, 19 equations, 7 figures, 9 tables)

This paper contains 20 sections, 19 equations, 7 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Traditional Methods
  4. Convolutional Neural Networks
  5. Vision Transformer
  6. State Space Models
  7. Learning With Noisy Labels
  8. Method
  9. Overall Architecture
  10. DSDFormer Block
  11. Temporal Reasoning Confident Learning (TRCL)
  12. Experiment
  13. Dataset
  14. Implementation Details
  15. Evaluation Metrics
  16. ...and 5 more sections

Figures (7)

  • Figure 1: Vision-based driver distraction detection employed in intelligent transportation systems. The design of the figure is inspired by nan-tiv.
  • Figure 2: Compared to traditional attention mechanisms, the Mamba architecture offers advantages in time complexity.
  • Figure 3: DSDFormer comprises the stem, four stages, and the projection head, where the stage stacks several DSDFormer Blocks sequentially. DSDFormer Block consists of a dual state domain attention (DSDA), a spatial-channel enhancement module (SCEM), a multi-branch enhancement module (MBEM), and a lightweight feed-sforward network (LFFN). Conv and DW Conv refers to the convolution and depth-wise convolution, respectively. Linear refers to the fully connected operation, and AvgPool refers to the average pooling operation.
  • Figure 4: Some examples of noisy labels are illustrated.
  • Figure 5: We visualized the noise cleaning effect between TRCL and CL on the AUC-V1. TRCL achieves lower noise rates for each category.
  • ...and 2 more figures