Driving as a Diagnostic Tool: Scenario-based Cognitive Assessment in Older Drivers from Driving Video

TL;DR

This work tackles the underdiagnosis of cognitive decline in older adults by proposing a scenario-based cognitive assessment that leverages naturalistic driving videos and large vision models to extract digital biomarkers of impairment. The authors introduce a two-phase framework: (1) scenario-specific video sampling from a curated driving dataset (RWRAD) and (2) a vision-transformer–driven representation pipeline followed by dimensionality reduction and scenario ranking to identify the most discriminative driving contexts. By training two identical RF classifiers on the most and least discriminative scenarios, the approach demonstrates robust separation between Normal-aging and MD-aging groups, achieving up to 72.32% accuracy and 80.13% F1 in the most informative freeway-interchange context under driver-level separation. The results suggest that scenario-focused, non-invasive monitoring of real-world driving can serve as a scalable biomarker for early cognitive decline and disease progression, with potential to complement traditional cognitive testing and extend monitoring outside clinical settings.

Abstract

We introduce scenario-based cognitive status identification in older drivers from naturalistic driving videos, leveraging large vision models. In recent times, cognitive decline including Dementia and Mild Cognitive Impairment (MCI), is often underdiagnosed due to the time-consuming and costly nature of current diagnostic methods. By analyzing real-world driving behavior captured through in-vehicle sensors, this study aims to extract "digital fingerprints" that correlate with functional decline and clinical features of dementia. Moreover, modern large vision models can draw meaningful insights from everyday driving patterns across different roadway scenarios to early detect cognitive decline. We propose a framework that uses large vision models and naturalistic driving videos to analyze driver behavior, identify cognitive status and predict disease progression. We leverage the strong relationship between real-world driving behavior as an observation of the current cognitive status of the drivers where the vehicle can be utilized as a "diagnostic tool". Our method identifies early warning signs of functional impairment, contributing to proactive intervention strategies. This work enhances early detection and supports the development of scalable, non-invasive monitoring systems to mitigate the growing societal and economic burden of cognitive decline in the aging population.

Figures (10)

• Figure 1: Scenario-based cognitive assessment framework. The pipeline consists of two phases - scenario selection and scenario validation. For testing, the fine-tuned RF model that produced the highest performance was used.
• Figure 2: Breakdown of the recorded driving data in the RWRAD dataset.
• Figure 3: Cognitive score distribution across all subjects ($N = 69$). (a) COGSTAT score, (b) MoCA score. We combined the MD and MCI subjects into a unified class "MD-aging" and performed two-class classification with the "Normal-aging".
• Figure 4: Geospatial map of the OMR database (Omaha, NE). The purple lines indicate the routes where the participants drove most frequently during the RWRAD data collection period.
• Figure 5: Scenario-specific video sampling. The top row represents the rollout of all valid drives and driving videos across subjects in the OMR data. From each subject $D_i$, valid drives {$dr_1, dr_2, \dots, dr_n$} and their associated video samples {$V_1, V_2, \dots, V_m$} were mapped into scenario-specific subsets. These video samples were aggregated and used as input to the large vision model for driving pattern analysis.