Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VTD: Visual and Tactile Database for Driver State and Behavior Perception

Jie Wang, Mobing Cai, Zhongpan Zhu, Hongjun Ding, Jiwei Yi, Aimin Du

TL;DR

This work tackles the safety challenges of human-vehicle co-driving by addressing the paucity of comprehensive, multimodal datasets for driver fatigue and distraction. It introduces VTD, a long-sequence, visual-tactile database collected in a driving-simulation platform, combining RGB frontal video, eye-tracking, tactile ECG via steering-wheel electrodes, and vehicle signals. The dataset includes data from 15 fatigued participants and 17 takeover cases across 102 takeover experiments, with synchronized, labeled time-series and raw multimodal recordings. Key contributions include multi-view video capture via Tobii glasses, a non-intrusive tactile ECG sensing method, and data processing pipelines for normalization, feature extraction, and cross-modal fusion, positioning VTD as a standardized resource for benchmarking cross-modal driver perception algorithms. The dataset is poised to improve fatigue and distraction detection, driver-state estimation, and safe takeover strategies in intelligent driving systems, with immediate implications for safety in human-in-the-loop co-driving scenarios.

Abstract

In the domain of autonomous vehicles, the human-vehicle co-pilot system has garnered significant research attention. To address the subjective uncertainties in driver state and interaction behaviors, which are pivotal to the safety of Human-in-the-loop co-driving systems, we introduce a novel visual-tactile perception method. Utilizing a driving simulation platform, a comprehensive dataset has been developed that encompasses multi-modal data under fatigue and distraction conditions. The experimental setup integrates driving simulation with signal acquisition, yielding 600 minutes of fatigue detection data from 15 subjects and 102 takeover experiments with 17 drivers. The dataset, synchronized across modalities, serves as a robust resource for advancing cross-modal driver behavior perception algorithms.

VTD: Visual and Tactile Database for Driver State and Behavior Perception

TL;DR

This work tackles the safety challenges of human-vehicle co-driving by addressing the paucity of comprehensive, multimodal datasets for driver fatigue and distraction. It introduces VTD, a long-sequence, visual-tactile database collected in a driving-simulation platform, combining RGB frontal video, eye-tracking, tactile ECG via steering-wheel electrodes, and vehicle signals. The dataset includes data from 15 fatigued participants and 17 takeover cases across 102 takeover experiments, with synchronized, labeled time-series and raw multimodal recordings. Key contributions include multi-view video capture via Tobii glasses, a non-intrusive tactile ECG sensing method, and data processing pipelines for normalization, feature extraction, and cross-modal fusion, positioning VTD as a standardized resource for benchmarking cross-modal driver perception algorithms. The dataset is poised to improve fatigue and distraction detection, driver-state estimation, and safe takeover strategies in intelligent driving systems, with immediate implications for safety in human-in-the-loop co-driving scenarios.

Abstract

In the domain of autonomous vehicles, the human-vehicle co-pilot system has garnered significant research attention. To address the subjective uncertainties in driver state and interaction behaviors, which are pivotal to the safety of Human-in-the-loop co-driving systems, we introduce a novel visual-tactile perception method. Utilizing a driving simulation platform, a comprehensive dataset has been developed that encompasses multi-modal data under fatigue and distraction conditions. The experimental setup integrates driving simulation with signal acquisition, yielding 600 minutes of fatigue detection data from 15 subjects and 102 takeover experiments with 17 drivers. The dataset, synchronized across modalities, serves as a robust resource for advancing cross-modal driver behavior perception algorithms.

Paper Structure

This paper contains 17 sections, 2 equations, 2 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Datasets for Driver State
  4. Datasets for Driver Behaviors
  5. METHODOLOGY
  6. Overall Framework
  7. Fatigue Driving Experiment
  8. Distracted Driving and Takeover Experiment
  9. VTD Experiment Setup Innovation
  10. Multi-channel and Multi-angle Videos
  11. Tactile sensing device for driver’s ECG
  12. Data Processing Method Innovations
  13. PROPERTIES
  14. Video Data Characteristics and Applications
  15. Properties and Functions of Tactile Data
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Research Gap in Driver Behavior and Driver State
  • Figure 2: VTD Data Collection Infrastructure